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Preface 


This  monograph  is  one  of  a  series  from  a  RAND  Project  AIR 
FORCE  (PAF)  project,  “The  Cost  of  Future  Military  Aircraft:  His¬ 
torical  Cost  Estimating  Relationships  and  Cost  Reduction  Initia¬ 
tives.”  The  purpose  of  the  project  is  to  improve  the  tools  used  to  es¬ 
timate  the  costs  of  future  weapon  systems.  It  focuses  on  how  recent 
technical,  management,  and  government  policy  changes  affect  cost. 
This  document  reports  research  findings  on  lessons  learned  relevant 
to  the  Department  of  Defense  (DoD)  cost  estimating  and  system  ac¬ 
quisition  management  community  related  to  implementing  evolu¬ 
tionary  acquisition  (EA)  strategies  in  major  defense  space  acquisition 
programs.  EA  strategies  are  major  acquisition  reform  measures  that 
were  formalized  as  the  “preferred  approach”  to  weapon  system  acqui¬ 
sition  under  the  first  Bush  administration’s  DoD  leadership.  While 
these  strategies  aim  at  enhancing  the  outcomes  of  all  aspects  of  the 
acquisition  process,  their  single  most  important  objective  is  to  reduce 
dramatically  the  time  between  the  identification  of  new  operational 
needs  and  the  fielding  of  operationally  useful  equipment  to  begin  to 
meet  those  needs. 

The  core  concept  of  EA  strategies  is  to  acquire  new  capabilities 
in  multiple,  shorter-phased  spirals  or  increments.  In  principle,  the 
initial  spirals  or  increments  provide  a  basic  “threshold”  capability 
relatively  quickly,  which  is  operationally  useful.  Subsequent  spirals  or 
increments  build  on  this  to  provide  more  capability,  eventually  re¬ 
sulting  in  a  system  that  meets  the  full  “objective”  capability  originally 
envisioned  at  the  beginning  of  the  program.  This  approach  contrasts 
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with  the  traditional  acquisition  strategy  of  focusing  on  a  single  step  to 
full  objective  capability,  an  approach  that  critics  claim  often  results  in 
inordinately  long  developmental  schedules  that  produce  no  useful 
operational  capability  for  many  years,  and  that  often  lead  to  serious 
problems  with  schedule  slippage  and  cost  growth.  Thus,  EA  strategies 
are  intended  to  increase  the  efficiency  of  DoD’s  acquisition  process 
and  result  in  the  rapid  fielding  of  useful  new  capabilities.  This  re¬ 
search  aims  to  examine  some  of  the  implications  of  the  EA  strategy 
for  the  cost  estimating  and  acquisition  management  community. 

These  research  findings  are  based  on  extensive  structured  inter¬ 
views  with  government  cost  analysts,  contracting  officers,  and  other 
senior  acquisition  officials  representing  the  Office  of  the  Secretary  of 
Defense  (OSD)  and  the  Office  of  the  Secretary  of  the  Air  Force 
(SAF).  For  more  detailed  lessons  learned  in  implementing  EA  strate¬ 
gies,  the  authors  interviewed  the  relevant  officials  from  several  pro¬ 
gram  offices  at  the  U.S.  Air  Force  Space  and  Missile  Systems  Center 
(SMC)  and  the  DoD  Missile  Defense  Agency  (MDA) . 1 

This  monograph  should  be  of  interest  to  cost  analysts,  con¬ 
tracting  officers,  acquisition  policymakers,  and  other  senior  acquisi¬ 
tion  officials  interested  in  acquisition  reform  and  new  approaches  to 
contracting  and  incentivizing  contractors  to  achieve  the  best  value  in 
defense  procurement. 

Other  RAND  Project  AIR  FORCE  documents  that  address 
military  aircraft  and  other  cost  estimating  issues  include  these: 

•  In  An  Overview  of  Acquisition  Reform  Cost  Savings  Estimates, 
MR-1329-AF,  Mark  A.  Forell  and  John  C.  Graser  used  relevant 
literature  and  interviews  to  determine  whether  estimates  of  the 
efficacy  of  acquisition  reform  measures  are  robust  enough  to  be 
of  predictive  value. 

•  In  Military  Airframe  Acquisition  Costs :  The  Effects  of  Lean 
Manufacturing,  MR-1325-AF,  Cynthia  R.  Cook  and  John  C. 
Graser  examine  the  package  of  new  tools  and  techniques  known 


1  See  Chapter  One  for  organizations  represented  by  interviewees. 
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as  “lean  production”  to  determine  whether  it  would  enable  air¬ 
craft  manufacturers  to  produce  new  weapon  systems  at  costs 
below  those  predicted  by  historical  cost  estimating  models. 

In  Military  Airframe  Costs:  The  Effects  of  Advanced  Materials  and 
Manufacturing  Processes ,  MR-1370-AF,  Obaid  Younossi,  Mi¬ 
chael  Kennedy,  and  John  C.  Graser  examine  cost  estimating 
methodologies  and  focus  on  military  airframe  materials  and 
manufacturing  processes.  This  monograph  provides  cost  estima¬ 
tors  with  factors  useful  in  adjusting  and  creating  estimates  based 
on  parametric  cost  estimating  methods. 

In  Military  Jet  Engine  Acquisition :  Technology  Basics  and  Cost- 
Estimating  Methodology,  MR-1596-AF,  Obaid  Younossi,  Mark 
V.  Arena,  Richard  M.  Moore,  Mark  A.  Lorell,  Joanna  Mason, 
and  John  C.  Graser  present  a  new  methodology  for  estimating 
military  jet  engine  costs  and  discuss  the  technical  parameters 
that  derive  the  engine  development  schedule,  development  cost, 
and  production  costs,  and  present  quantitative  analysis  of  his¬ 
torical  data  on  engine  development  schedule  and  cost. 

In  Test  and  Evaluation  Trends  and  Costs  in  Aircraft  and  Guided 
Weapons,  MG-109-AF,  Bernard  Fox,  Michael  Boito,  John  C. 
Graser,  and  Obaid  Younossi  examine  the  effects  of  changes  in 
the  test  and  evaluation  (T&E)  process  used  to  evaluate  military 
aircraft  and  air-launched  guided  weapons  during  their  develop¬ 
ment  programs. 

In  Software  Cost  Estimation  and  Sizing  Methods:  Issues,  and 
Guidelines,  MG-269-AF,  Shari  Lawrence  Pfleeger,  Felicia  Wu, 
and  Rosalind  Lewis  recommend  an  approach  to  improve  the 
utility  of  the  software  cost  estimates  by  exposing  uncertainty  and 
reducing  risks  associated  with  developing  the  estimates. 

In  Lessons  Learned  from  the  F/A-22  and  FI  A- 18  E/F  Development 
Programs,  MG-276-AF,  Obaid  Younossi,  David  E.  Stem,  Mark 
A.  Lorell,  and  Frances  M.  Lussier  evaluate  historical  cost,  sched¬ 
ule,  and  technical  information  from  development  of  the  F/A-22 
and  F/A-18  E/F  programs  to  derive  lessons  for  the  Air  Force  and 
other  services  to  improve  the  acquisition  of  future  systems. 
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The  research  reported  here  was  sponsored  by  Lieutenant  Gen¬ 
eral  John  D.  W.  Corley,  Principal  Deputy  Assistant  Secretary  of  the 
Air  Force  for  Acquisition  (SAF/AQ),  and  conducted  within  the  Re¬ 
source  Management  Program  of  RAND  Project  AIR  FORCE.  The 
point  of  technical  contact  was  Mr.  Jay  Jordan,  Technical  Director, 
Air  Force  Cost  Analysis  Agency  (AFCAA/TD). 


RAND  Project  AIR  FORCE 

RAND  Project  AIR  FORCE  (PAF),  a  division  of  the  RAND  Corpo¬ 
ration,  is  the  U.S.  Air  Force’s  federally  funded  research  and  develop¬ 
ment  center  for  studies  and  analyses.  PAF  provides  the  Air  Force  with 
independent  analyses  of  policy  alternatives  affecting  the  development, 
employment,  combat  readiness,  and  support  of  current  and  future 
aerospace  forces.  Research  is  conducted  in  four  programs:  Aerospace 
Force  Development;  Manpower,  Personnel,  and  Training;  Resource 
Management;  and  Strategy  and  Doctrine. 

Additional  information  about  PAF  is  available  on  our  Web  site 
at  http://www.rand.org/paf. 
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Summary 


Introduction  and  Research  Approach 

This  monograph  presents  findings  of  a  RAND  Project  AIR  FORCE 
research  project  that  documented  the  lessons  learned  by  the  U.S.  Air 
Force  and  other  Department  of  Defense  (DoD)  cost  analysis  and  ac¬ 
quisition  community  members  from  the  implementation  of  evolu¬ 
tionary  acquisition  (EA)  strategies  for  major  Air  Force  defense  space 
acquisition  programs.  In  May  2003,  DoD  promulgated  revised  5000 
series  acquisition  directives  and  instructions  that  mandated  EA  strate¬ 
gies  relying  on  the  spiral  development  process  as  the  preferred  ap¬ 
proach  to  satisfying  operational  needs.1  These  same  concepts  were 
later  incorporated  into  a  new  space  acquisition  policy  document,  the 
National  Security  Space  Acquisition  Policy  (NSSAP)  03-01  (DoD, 
2004). 

The  principal  goal  of  EA  strategies  is  to  provide  operationally 
useful  capabilities  to  the  warfighter  much  more  quickly  than  tradi¬ 
tional  acquisition  strategies.  Instead  of  the  old  approach  of  “single 
step  to  full  capability,”  evolutionary  acquisition  aims  at  achieving  an 
overall  objective  end  capability  through  the  more  rapid  fielding  of 
numerous  operationally  useful  threshold  capabilities  by  pursuing  less 
demanding  intermediary  increments  or  steps.  In  theory,  the  initial 
spirals  or  increments  provide  a  basic  “threshold”  capability  relatively 


1  The  DoD  5000  acquisition  policy  guidance  documents  have  been  the  key  DoD  publica¬ 
tions  establishing  the  basic  process  and  structure  for  developing  and  procuring  U.S.  weapon 
systems  since  July  1971  (Ferrara,  1996). 


xv 


xvi  Evolutionary  Acquisition 


quickly,  which  is  operationally  useful  to  the  user.  Subsequent  spirals 
or  increments  build  on  this  to  provide  more  capability,  eventually 
resulting  in  a  system  that  meets  the  full  “objective”  capability  origi¬ 
nally  envisioned  at  the  beginning  of  the  program.2 3 

Spiral  development,  the  preferred  process  for  implementing  EA, 
suggests  an  even  more  radical  departure  from  past  approaches,  be¬ 
cause  it  envisions  an  ongoing  process  of  evolutionary  development  of 
the  system  requirements,  based  on  feedback  loops  from  warfighters 
using  field  demonstrations,  and  from  other  stakeholders.  As  explained 
in  the  official  DoD  5000  series  guidance, 

In  this  process  (Spiral  Development),  a  desired  capability  is 
identified,  but  the  end-state  requirements  are  not  known  at  pro¬ 
gram  initiation.  Those  requirements  are  refined  through  demon¬ 
stration  and  risk  management;  there  is  continuous  user  feedback-, 
and  each  increment  provides  the  user  the  best  possible  capabil¬ 
ity.  The  requirements  for  future  increments  depend  on  feedback 
from  users  and  technology  maturation } 

EA  advocates  claim  many  potential  benefits  from  the  adoption 
of  the  strategy,  while  skeptics  raise  numerous  concerns  about  formi¬ 
dable  barriers  and  challenges  to  implementing  the  policy  effectively. 
Yet  little  documented  objective  experience  and  evidence  exist  to  assess 
the  policy  and  very  little  systematic  analysis  of  what  evidence  does 
exist  has  yet  been  published.  Nonetheless,  almost  all  agreed  at  the 
time  of  its  formal  adoption  that  full  implementation  of  EA  using  the 


2  The  authors  are  aware  of  no  authoritative  official  DoD  definition  of  “objective”  and 
“threshold”  capabilities.  However,  based  on  the  implied  meaning  of  the  words  in  common 
DoD  usage,  the  authors  provide  the  following  definitions.  Objective  capabilities  are  the  full 
end-state  capabilities  envisioned  for  a  new  weapon  system  at  the  beginning  of  an  acquisition 
program.  Objective  capabilities  may  be  attained  through  a  single-step-to-capability  tradi¬ 
tional  program,  or  through  an  EA  program  that  goes  through  the  fielding  of  several  interim 
systems  that  meet  useful  but  lower  threshold  capabilities.  Threshold  capabilities  are  the 
minimum  capabilities  thought  necessary  to  justify  development  of  a  new  system  or  a  new 
variant  of  the  system. 

3  DoD  (2003c).  Emphasis  added.  The  other  process  identified  for  implementing  evolution¬ 
ary  acquisition  is  called  incremental  development.  With  incremental  development,  the  end- 
state  requirements  are  known.  This  is  an  important  conceptual  distinction  that  is  discussed  at 
length  in  the  body  of  this  monograph. 
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spiral  development  process  by  DoD  would  necessitate  a  major  over¬ 
haul  of  DoD  acquisition  procedures,  particularly  in  the  areas  of  re¬ 
quirements  management,  budgeting,  cost  analysis,  and  elsewhere. 

The  overarching  objective  of  this  RAND  research  effort  is  to  as¬ 
sist  cost  analysts  and  other  elements  of  the  Air  Force  acquisition  man¬ 
agement  community  in  formulating  cost  analysis  and  program  man¬ 
agement  policies  and  procedures  that  anticipate  and  respond  to  the 
prospect  of  more  widespread  use  of  EA  strategies.4 

This  research  effort  adopted  a  three-pronged  approach.  First,  the 
project  team  carried  out  a  comprehensive  review  of  published  and 
unpublished  reports  and  other  studies  on  the  theory  and  implementa¬ 
tion  of  evolutionary  acquisition.  Second,  it  conducted  a  wide-ranging 
series  of  interviews  with  senior  DoD  and  Air  Force  acquisition  man¬ 
agement  officials  regarding  their  understanding  of  the  meaning  and 
implications  of  DoD’s  mandated  EA  policies.  Finally,  it  carefully  re¬ 
viewed  five  major  space  acquisition  programs  that  have  been  recently 
restructured  in  accordance  with  EA  concepts,  in  order  to  gain  lessons 
learned  to  date  on  the  implementation  of  EA,  particularly  in  areas  of 
relevance  for  cost  analysts.  The  information  on  these  case  studies  was 
derived  from  open  sources  and  from  interviews  with  senior  program 
officials.5  These  are  the  five  case  studies  reviewed: 

•  Space-Based  Space  Surveillance  (SBSS)  System 

•  Rapid  Attack  Identification,  Detection,  and  Reporting  System 
(RAIDRS) 

•  Global  Positioning  Satellite  (GPS)  III 

•  Space-Based  Radar  (SBR) 

•  Kinetic  Energy  Interceptor  (KEI)6 


4  This  study,  however,  focuses  largely  on  the  implications  of  EA  for  the  cost  analysis  com¬ 
munity. 

5  The  interviews  were  conducted  by  the  authors  and  took  place  primarily  during  the  first 
nine  months  of  2004. 

®  KEI  is,  of  course,  a  Missile  Defense  Agency  program.  However,  it  was  widely  recom¬ 
mended  to  us  by  DoD  officials  as  an  instructive  example  of  one  innovative  approach  to  im¬ 
plementing  EA. 
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The  authors  chose  to  focus  on  major  space  programs  for  two 
reasons.  First,  space  programs  have  recently  grown  significantly  in 
relative  importance  for  the  future  of  the  Air  Force,  and  are  central  to 
DoD’s  plans  for  transformation  of  the  U.S.  armed  forces.  Second,  the 
Air  Force  has  recently  completed  a  thoroughgoing  review  and  com¬ 
plete  overhaul  of  space  acquisition  policy.  The  new  National  Security 
Space  Acquisition  Policy  (NSSAP)  03-01  guidance  mandates  evolu¬ 
tionary  acquisition  as  the  preferred  acquisition  approach  for  space 
programs.  Most  major  new  Air  Force  space  programs  have  been  struc¬ 
tured,  at  least  initially,  as  evolutionary  acquisition  programs  based  on 
spiral  development.7 

The  programs  examined  as  case  studies  are  all  in  the  very  earliest 
stages  of  the  acquisition  process.  Therefore,  the  lessons  learned  de¬ 
rived  from  them  must  be  considered  tentative  and  treated  as  provi¬ 
sional.  In  addition,  it  is  important  to  note  that  the  programs  varied  in 
size  and  complexity,  as  well  as  in  the  interpretation,  definition,  and 
application  of  the  rather  general  guidance  on  the  evolutionary  acqui¬ 
sition  strategies  and  processes  provided  by  the  DoD  5000  series  and 
NSSAP  03-01.  The  authors  provide  a  taxonomy  of  the  different  gen¬ 
eral  types  of  programs  in  the  list  of  case  studies.  (See  pp.  39-40, 
45-46.) 

The  next  section  summarizes  the  EA  implementation  lessons  de¬ 
rived  from  these  case  studies,  as  well  as  overall  findings  for  acquisition 
managers  and  for  cost  analysts. 


EA  Implementation  Lessons  Learned  from  Case  Studies 

Many  of  the  issues,  findings,  and  recommendations  mentioned  in 
this  section  are  applicable  to  all  major  DoD  defense  acquisition  pro¬ 
grams,  not  only  those  conducted  according  to  the  EA  process. 


7  See  Chapter  One  for  organizations  represented  by  interviewees.  The  applicability  to 
nonspace  programs  of  lessons  learned  derived  from  the  use  of  EA  on  space  programs  is  an 
area  of  some  contention.  The  authors  review  the  relevant  evidence  in  the  body  of  this  study, 
and  conclude  that  the  EA  lessons  learned  from  space  programs  are  largely  applicable  to  other 
types  of  major  DoD  system  acquisition  programs. 
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However,  there  was  a  wide  consensus  among  those  interviewed  that 
EA  promotes  certain  program  characteristics  that  make  many  of  these 
issues  and  challenges  more  prominent. 

EA  programs  require  considerable  additional  up-front  man¬ 
agement  planning  and  engineering  workload  and  the  budget 
sources  to  support  them.  (See  below  and  Chapter  Two,  especially 
pp.  51,  88.)  This  situation  arises  from  the  necessity  to  map  out  the 
complex  program  structure  implied  by  EA,  which  includes  a  series  of 
separate,  overlapping  increments,  each  requiring  the  definition  of  op¬ 
erationally  useful  threshold  and  objective  requirements,  and  each  in¬ 
cluding  formal  milestone  requirements  mimicking  a  stand-alone  pro¬ 
gram;  and  the  additional  system  engineering  and  nonrecurring 
engineering  to  support  the  progression  of  upgrades  and  technology 
insertion  that  take  place  within  and  between  increments.  Also,  addi¬ 
tional  up-front  resources  are  required  to  support  more  extensive  and 
continuously  revised  evolutionary  costing  efforts. 

EA  programs  using  the  spiral  development  process  should 
focus  on  capability  mission  objectives  rather  than  traditional 
technical  requirements.  (See  Chapter  Two,  especially  pp.  46-48, 
84-85.)  With  the  spiral  development  process,  the  system  require¬ 
ments  emerge  and  evolve  over  time.  Therefore,  the  program  focus, 
particularly  at  program  inception,  must  be  on  capability  objectives. 
This  is  a  challenging  approach,  however,  given  the  current  acquisition 
regulatory  and  oversight  environment;  as  a  result,  most  programs  ex¬ 
amined  while  researching  this  monograph  are  tending  to  move  away 
from  spiral  development  and  toward  an  incremental  development 
process.8 

After  the  initial  formulation  and  stabilization  of  require¬ 
ments  at  the  beginning  of  a  program  or  program  increment  (a 
process  that  does  and  should  include  significant  feedback  from 
the  user  communities),  practical  implementation  of  EA  using  the 
spiral  development  approach  requires  a  more  structured  man¬ 
agement  of  the  user  community  feedback  loop  process.  (See 


8  Unlike  spiral  development,  the  incremental  development  process  assumes  that  the  end- 
state  requirements  are  known  at  the  inception  of  the  development  process. 
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Chapter  One,  pp.  22-24,  and  Chapter  Two,  especially  pp.  83-84.) 
Initial  conceptual  versions  of  EA  strategies  using  spiral  development 
envisioned  constantly  functioning  feedback  loops  from  the  user 
community  to  fine-tune  requirements  and  make  sure  that  developers 
produce  end  products  that  meet  real  needs  in  the  field.  But  actual  ex¬ 
perience  in  the  early  phases  of  the  case  study  programs  examined  sug¬ 
gests  that  undisciplined  feedback  in  the  early  concept  development 
stages,  particularly  on  programs  with  multiple  user  communities  such 
as  SBR,  can  lead  to  a  counterproductive  piling  up  of  sometimes  mu¬ 
tually  inconsistent  requirements,  concepts,  and  technologies,  and 
contributes  to  the  challenge  of  controlling  requirements  creep. 

Program  managers  believe  that  the  use  of  EA  on  their  pro¬ 
grams  will  result  in  numerous  different  variants  of  the  same  sys¬ 
tem.  This  situation  is  expected  to  complicate  logistics  planning 
and  implementation  greatly.  Plans  will  have  to  be  made  to  up¬ 
date,  retrofit,  or  dispose  of  earlier  versions.  (See  Chapter  Two,  es¬ 
pecially  p.  84.)  While  none  of  the  programs  examined  was  nearly  ad¬ 
vanced  enough  to  turn  out  operationally  deployable  hardware,  it  was 
the  consensus  view  among  program  managers  that  the  use  of  EA 
would  produce  multiple  variants  and  greatly  complicate  support 
planning  and  implementation.  Cost  analysts  also  noted  that  this 
problem  would  make  life-cycle  cost  (LCC)  estimating  more  difficult. 

The  next  section  summarizes  the  broader  overall  findings  from 
the  review  of  the  relevant  published  and  unpublished  literature,  inter¬ 
views  with  senior  DoD  acquisition  managers,  and  case  study  analysis. 
The  first  subsection  below  summarizes  general  findings  for  acquisi¬ 
tion  management.  The  second  subsection  below  summarizes  the 
authors’  overall  findings  for  cost  analysts. 


Summary  of  Overarching  Acquisition  Management 
Findings 

The  new  DoD  guidance  regarding  EA  (DoD  5000  series  and 
NSSAP  03-01)  permits  great  flexibility,  but  does  not  eliminate 
conceptual  and  definitional  ambiguity.  As  a  result,  EA  programs 
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vary  considerably  in  their  practical  implementation  approaches. 

(See  Chapter  Two,  and  Chapter  Three,  especially  pp.  89-90.) 9  The 
persistence  of  definitional  ambiguity  and  continuing  lack  of  precise 
implementation  guidelines  have  led  to  a  range  of  differing  interpreta¬ 
tions  of  key  terms  and  concepts  in  the  structuring  of  EA  programs. 
Such  key  EA  concepts  as  feedback  loops,  and  the  difference  between 
the  spiral  and  incremental  development  processes,  remain  areas  of 
debate  and  confusion  in  various  sectors  of  the  acquisition  manage¬ 
ment  community.  Because  of  the  serious  implementation  challenges 
posed  by  EA,  especially  using  the  preferred  approach  of  spiral  devel¬ 
opment,  DoD  should  further  clarify  its  guidance  for  implementing 
EA  programs. 

All  of  the  case  studies  point  to  the  conclusion  that  the  capa¬ 
bilities  and  requirements  definition  and  management  processes 
are  major  challenges  in  all  EA  programs.  Appropriate  structuring 
of  EA  phases  with  operationally  useful  threshold  requirements 
and  mapping  the  path  to  overall  objective  capability  are  de¬ 
manding  tasks  on  most  EA  programs.  (See  Chapter  Two,  especially 
pp.  46-48,  pp.  84-85.)  This  is  particularly  the  case  in  large,  complex 
hardware  procurement  programs  using  the  spiral  development  proc¬ 
ess,  where  the  objective  end  requirements  are  not  known  at  the  incep¬ 
tion  of  the  program.  Areas  of  particular  concern  include  the  issues  of 
structuring  specific  spirals  or  increments,  defining  operationally  use¬ 
ful  threshold  requirements  for  specific  spirals  or  increments,  and 
mapping  out  the  specific  path  to  the  objective  end  requirements.  Pre¬ 
venting  requirements  creep  is  perceived  as  a  significant  challenge. 

The  use  of  the  officially  preferred  spiral  development  process 
for  implementing  EA  on  major  hardware  acquisition  programs 
greatly  increases  the  level  of  program  uncertainties,  raising  seri¬ 
ous  challenges  for  program  managers  in  the  current  acquisition 
environment.  (See  Chapter  Two,  and  Chapter  Three,  pp.  91-93.) 


9  Definitional  ambiguity  and  lack  of  precise  guidance  have  permitted  not  only  significantly 
different  approaches  to  be  used  on  EA  programs,  but  have  also  permitted  certain  policies  that 
run  counter  to  the  original  intent  of  EA  advocates.  Chapters  Two  and  Three  expound  on 
this. 
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The  very  uncertainties  that  provide  acquisition  managers  with  valu¬ 
able  flexibility  necessary  to  gain  the  expected  benefits  from  EA 
through  spiral  development  also  raise  considerable  challenges  for 
managers  in  the  existing  acquisition  environment.  Acquisition  man¬ 
agers  of  large,  complex  hardware  acquisition  programs  report  that 
they  are  subjected  to  very  strong  pressures  from  political  authorities, 
as  well  as  from  the  requirements  and  cost  analysis  communities,  to 
provide  far  more  detail  about  the  end  stages  of  the  program  than  they 
believe  is  feasible  under  the  spiral  development  approach.  As  a  result, 
many  programs  that  started  with  the  spiral  development  approach 
have  been  evolving  toward  an  incremental  acquisition  approach,  or 
something  more  akin  to  a  traditional  single-step-to-capability  pro¬ 
gram  with  planned  upgrades. 

Therefore,  the  authors  believe  that  evolutionary  acquisition  us¬ 
ing  the  preferred  approach  of  spiral  development,  as  laid  out  in  the 
most  recent  DoD  5000  and  NSSAP  03-01  guidance,  cannot  likely  be 
realistically  implemented  in  the  current  political  and  acquisition  envi¬ 
ronment  on  major  DoD  space  programs,  and  even  perhaps  on  other 
large-scale  DoD  hardware  acquisition  programs.  At  best,  EA  using 
spiral  development  may  be  one  useful  tool  that  can  be  used  in  some 
limited  circumstances  on  software  programs,  on  smaller-scale  hard¬ 
ware  programs,  or  perhaps  on  programs  such  as  KEI  that  operate  out¬ 
side  the  traditional  acquisition  framework. 


Summary  of  Cost  Management  Findings 

EA  programs  require  an  evolutionary  costing  approach.  By  neces¬ 
sity,  EA  using  the  spiral  development  approach  generally  tends  to 
lead  to  a  heavy  focus  of  the  cost  analysis  effort  on  the  initial  spiral 
or  increment,  at  the  expense  of  other  phases  of  the  program.  (See 
Chapter  Two,  especially  pp.  49-50,  86-87.)  Program  managers  and 
other  acquisition  managers  interviewed  noted  that  evolutionary  cost¬ 
ing  is  the  only  feasible  and  realistic  approach  to  use  on  EA  programs, 
especially  those  that  employ  the  spiral  development  process.  Evolu¬ 
tionary  costing  requires  that  the  cost  analysts  work  closely  with  gov- 
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ernment  engineers  for  independent  assessment  of  technological  risk 
and  schedule  of  the  program.  In  addition,  they  need  to  work  closely 
with  the  contractors  to  track  the  design  and  the  technologies,  and 
thus  the  costs,  as  they  evolve.  Cost  models  jointly  developed  by  the 
Program  Office  and  the  prime  contractor  are  often  used.  However, 
the  inputs  and  the  underlying  assumptions  to  the  cost  model  may  be 
different.  The  authors  found  virtual  unanimity  among  the  cost  ana¬ 
lysts  interviewed  on  all  levels  that  evolutionary  costing  is  feasible  and 
that  it  works  well,  when  government  and  contractor  cost  analysts 
work  closely  together  in  a  nonpoliticized  environment,  and  where 
general  agreement  has  been  reached  among  user  communities  on  sys¬ 
tem  requirements  and  the  associated  level  of  technological  risk. 

Overall,  most  cost  analysts  interviewed  expressed  generally 
positive  views  about  EA.  Nonetheless,  lingering  concerns  did  sur¬ 
face  during  the  interviews  regarding  a  variety  of  cost  issues  associ¬ 
ated  with  EA.  (See  Chapter  Two,  especially  pp.  48-51,  86-87.) 
Some  of  those  concerns  are  listed  below. 

1.  Committing  the  U.S.  Air  Force  (USAF)  to  large,  costly  programs 
before  the  full  cost  implications  of  the  program  are  well  under¬ 
stood 

2.  Accurately  assessing  total  program  LCC,  support  costs,  and  retro¬ 
fit  costs  based  on  sound  and  independent  technical  assessment  of 
the  program  baseline 

3.  Adequately  budgeting  for  the  potentially  high  variability  of  cost 
outcomes  arising  from  the  high  degree  of  uncertainty  surrounding 
inputs  to  cost  models 

4.  Accurately  accounting  for  the  potential  cost  implications  of  re¬ 
quirements  creep  arising  from  multiple  users  and  planned  inser¬ 
tion  of  technologies  of  uncertain  future  maturity. 

A  strong  consensus  emerged  from  the  interviews  with  cost  ana¬ 
lysts  that  EA  is  an  important  and  useful  tool  providing  program  man¬ 
agers  with  useful  flexibility,  and  that  the  cost  community  can  ac¬ 
commodate  it  adequately  through  the  use  of  evolutionary  costing. 
Nonetheless,  there  was  recognition  at  least  among  some  of  those  in- 
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terviewed  that  EA  would  increase  the  cost  analyst’s  workload  and  re¬ 
quire  substantial  interface  with  the  engineers  and  the  contractor. 

EA  as  currently  defined  inherently  results  in  increased  uncertain¬ 
ties  regarding  technology  development  timelines  and  program  sched¬ 
ule  during  the  early  phases  of  a  program.  If  handled  with  care,  these 
increased  uncertainties  can  provide  opportunities  for  greater  program 
flexibility  and  program  realism.  At  the  same  time,  increased  program 
uncertainties  can  pose  difficult  political  and  budgetary  challenges  for 
program  managers  within  the  current  acquisition  environment. 
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AAC 

Air  Armament  Center 

ACAT 

Acquisition  Category 

ACE 

Acquisition  Center  of  Excellence 

ACTD 

Advanced  Concept  Technology  Demonstration 

AFCAA 

Air  Force  Cost  Analysis  Agency 

AFCAA/TD 

Technical  Director,  Air  Force  Cost  Analysis  Agency 

AFSPC 

Air  Force  Space  Command 

AoA 

analysis  of  alternatives 

AOC 

Air  Force  Operations  Center 

ASC 

Aeronautical  Systems  Center 

ASP 

Acquisition  Strategy  Panel 

BMC3 

Battlefield  Management,  Command,  Control,  and 
Communications 

BMDO 

Ballistic  Missile  Defense  Organization 

C2BMC 

command,  control,  battle  management,  and 
communications 

CAIG 

Cost  Analysis  Improvement  Group 

CAIV 

cost  as  an  independent  variable 

CD 

concept  decision 

CDD 

capability  development  document 

CDR 

critical  design  review 
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COA 

collaborative  operations  assessment 

CONOPS 

concept  of  operations 

COTS 

commercial  off-the-shelf 

CPD 

capability  production  document 

CR 

concept  refinement 

CTF 

combined  task  force 

D/OT&E 

developmental  and  operational  test  and  evaluation 

DAB 

Defense  Acquisition  Board 

DARPA 

Defense  Advanced  Research  Projects  Agency 

DAU 

Defense  Acquisition  University 

DCS 

Defensive  Counterspace 

DoD 

Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

DOTMLPF 

doctrine,  organization,  training,  materiel,  leadership 
and  education,  personnel  and  facilities 

DSCS 

Defense  Satellite  Communications  System 

DSP 

Defense  Support  Program  (satellites) 

DT&E 

developmental  test  and  evaluation 

EA 

evolutionary  acquisition 

ECP 

engineering  change  proposal 

ESA 

electronically  scanned  array 

ESC 

Electronic  Systems  Center 

FOC 

full  operational  capability 

FRP 

full-rate  production 

FY 

fiscal  year 

FYDP 

Future  Years  Defense  Plan 

GAO 

Government  Accountability  Office,  formerly  the 
General  Accounting  Office 

COTS 

government  off-the-shelf 
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GPS 

Global  Positioning  Satellite 

GRA 

Government  Reference  Architecture 

HALE 

High  Altitude  Long  Endurance 

HRTI 

high  resolution  terrain  information 

IC 

integrated  circuit 

ICA 

independent  cost  assessment 

ICAT 

independent  cost  analysis  team 

ICD 

initial  capabilities  document 

ICE 

independent  cost  estimate 

IDS 

interference  detection  system 

ILC 

initial  launch  capability 

IMPCS 

interference  monitoring  power  control  subsystem 

IOC 

initial  operational  capability 

IOT&E 

initial  operation  test  and  evaluation 

IPS 

integrated  program  summary 

JASSM 

joint  air-to-surface  strike  missile 

JCS 

Joint  Chiefs  of  Staff 

JDAM 

joint  direct  attack  munition 

JPO 

Joint  Program  Office 

JROC 

Joint  Requirements  Oversight  Council 

JSOW 

Joint  Stand-Off  Weapon 

JSTARS 

Joint  Surveillance  Target  Attack  Radar  System 

KDP 

key  decision  point 

KEI 

Kinetic  Energy  Interceptor 

KPP 

key  performance  parameter 

LCC 

life-cycle  cost 

LEO 

low  earth  orbit 

LRIP 

low-rate  initial  production 

MDA 

Missile  Defense  Agency 
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MDAP 

Major  Defense  Acquisition  Program 

MEO 

medium  earth  orbit 

MSX/SBV 

Midcourse  Space  Experiment  with  Space-Based 

Visible  Sensor 

NAVWAR 

navigational  warfare 

NGA 

National  Geospatial  Intelligence  Agency 

NMS 

National  Military  Strategy 

NRO 

National  Reconnaissance  Office 

NSS 

National  Space  Strategy 

NSSAP 

National  Security  Space  Acquisition  Policy 

O&S 

operations  and  support 

ORD 

operational  requirements  document 

OSD 

Office  of  the  Secretary  of  Defense 

OT&E 

Operational  Test  and  Evaluation 

OTA 

Other  Transaction  Authority 

P3I 

preplanned  product  improvement 

PAF 

Project  AIR  FORCE 

PBA 

price-based  acquisition 

PD 

production  and  deployment 

PDR 

preliminary  design  review 

PEO 

program  executive  office 

PRDA 

program  research  and  development  document 

R&D 

research  and  development 

RAIDRS 

Rapid  Attack  Identification,  Detection,  and  Reporting 
System 

RDT&E 

research,  development,  test,  and  evaluation 

S&T 

science  and  technology 

SAF 

Secretary  of  the  Air  Force 

SAF/AQ 

Secretary  of  the  Air  Force  for  Acquisition 

SAIC 

Science  Applications  International  Corporation 
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SAIV 

schedule  as  an  independent  variable 

SAR 

synthetic  aperture  radar 

SBIRS 

Space-Based  Infrared  System 

SBR 

Space-Based  Radar 

SBSS 

Space-Based  Space  Surveillance 

SBV 

space-based  visible  sensor 

S/C 

spacecraft 

SD 

spiral  development 

SDD 

system  development  and  demonstration 

SDIO 

Strategic  Defense  Initiative  Organization 

SDR 

system  design  review 

SMC 

Space  and  Missile  Systems  Center 

SMTI 

surface  moving  target  indication 

SPO 

system  program  office 

SR 

Space  Radar 

SRR 

system  requirements  review 

ss 

space  sensor 

SSA 
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CHAPTER  ONE 


Introduction:  The  Evolutionary  Acquisition 
Concept 


Overview 

This  monograph  presents  lessons  learned  on  the  implementation  of 
evolutionary  acquisition  (EA)  strategies  for  the  procurement  of  major 
space  defense  systems.  It  focuses  on  issues  of  interest  to  the  U.S.  Air 
Force  and  other  Department  of  Defense  (DoD)  cost  analysis  and  ac¬ 
quisition  management  communities. 

In  May  2003,  DoD  promulgated  its  long-anticipated  definitive 
revision  of  the  5000  series  acquisition  directives  and  instructions  that 
govern  the  acquisition  of  major  weapon  systems.1  These  documents 
mandated  EA  strategies  as  the  “preferred  approach”  to  satisfying  op¬ 
erational  needs.2  Formal  adoption  and  full  implementation  of  EA  by 
DoD  was  widely  viewed  at  the  time  as  necessitating  a  major  overhaul 
of  DoD  acquisition  procedures,  particularly  in  the  areas  of  require¬ 
ments  management,  budgeting,  cost  estimating,  and  elsewhere.3 

The  principal  goal  of  EA  strategies  is  to  provide  operationally 
useful  capabilities  to  the  warfighter  much  more  quickly  than  tradi¬ 
tional  acquisition  strategies.  Instead  of  the  old  approach  of  “single 


1  The  DoD  5000  acquisition  policy  guidance  documents  have  been  the  key  DoD  publica¬ 
tions  establishing  the  basic  process  and  structure  for  developing  and  procuring  U.S.  weapon 
systems  since  July  1971  (Ferrara,  1996). 

2  DoD  first  formally  endorsed  EA  as  the  preferred  acquisition  approach  in  the  DoD  5000 
revised  regulations  issued  in  October  2000,  but  the  definitive  versions  of  the  new  regulations 
were  not  published  until  May  2003.  See  below  for  further  discussion. 

3  For  example,  see  Slate  (2002)  and  Sylvester  and  Ferrara  (2003). 
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step  to  full  capability,”  evolutionary  acquisition  aims  at  achieving  an 
overall  objective  end  capability  through  the  more  rapid  fielding  of 
numerous  operationally  useful  interim  threshold  capabilities4  by  pur¬ 
suing  less  demanding  intermediary  increments  or  steps.  Thus,  DoD 
defines  EA  as  an  approach  that 

delivers  capability  in  increments,  recognizing,  up  front,  the  need 
for  future  capability  improvements.  The  objective  is  to  balance 
needs  and  available  capability  with  resources,  and  to  put  capa¬ 
bility  into  the  hands  of  the  user  quickly.  The  success  of  the  strat¬ 
egy  depends  on  consistent  and  continuous  definition  of  re¬ 
quirements,  and  the  maturation  of  technologies  that  lead  to 
disciplined  development  and  production  of  systems  that  provide 
increasing  capability  towards  a  materiel  concept.  (DoD,  2003c, 
Section  3.3.1) 

Although  reformers  have  advocated  various  types  of  incremental 
acquisition  strategies  for  years,  there  had  never  been  a  DoD-wide  im¬ 
plementation  of  such  a  strategy  prior  to  the  formal  adoption  of  EA  in 
October  2000  (with  final  revisions  and  clarifications  in  May  2003). 5 
EA  advocates  claim  many  potential  benefits  from  the  adoption  of  the 
strategy,  while  skeptics  raise  numerous  concerns  about  formidable 
barriers  and  challenges  to  effectively  implementing  the  policy.  Yet 
little  objective  experience  and  evidence  exists  to  assess  the  policy,  and 
very  little  systematic  analysis  of  what  evidence  does  exist  has  yet  been 
undertaken. 

The  overarching  objective  of  this  RAND  research  effort  is  to  as¬ 
sist  cost  analysts  and  other  elements  of  the  Air  Force  acquisition 


4  The  authors  are  aware  of  no  definitive  official  DoD  definition  of  “objective”  and  “thresh¬ 
old”  capabilities.  However,  based  on  the  implied  meaning  of  the  words  in  common  DoD 
usage,  this  monograph  provides  the  following  definitions.  The  “objective”  capability  is  the 
full  end  capability  sought  for  the  system  following  the  completion  of  all  development  as  laid 
out  in  the  operational  requirements  document  (ORD).  Objective  capability  may  be  achieved 
either  through  a  traditional  single-step-to-capability  program,  or  through  a  progression  of 
interim  programs,  each  of  which  meets  a  desired  “threshold”  capability.  Threshold  capability 
is  the  minimum  system  capability  thought  necessary  to  justify  the  development  of  a  new 
system  variant  or  system  increment. 

^  See  discussion  below. 
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management  community  in  formulating  cost  analysis  and  program 
management  policies  and  procedures  that  anticipate  and  respond  to 
the  prospect  of  more  widespread  use  of  EA  strategies.  To  meet  this 
objective,  this  project  aimed  to  answer  these  questions: 

•  What  are  the  status,  intent,  and  programmatic  implications  of 
evolutionary  acquisition  strategies  as  currently  envisioned  by 
senior  DoD  acquisition  policymakers? 

•  What  programmatic  lessons  learned  are  emerging  from  recent 
Air  Force  major  space  acquisition  programs  that  incorporate  key 
elements  of  evolutionary  acquisition  strategies  as  defined  in  the 
revised  DoD  5000  series  issued  in  May  2003  and  incorporated 
into  the  National  Security  Space  Acquisition  Policy  03-01 
(NSSAP  03-01)? 

•  What  new  policies  and  methods  (if  any)  should  the  Air  Force 
cost  analysis  and  acquisition  community  adopt  to  assist  most  ef¬ 
fectively  in  the  implementation  of  evolutionary  acquisition  on 
space  programs? 

In  order  to  provide  some  insight  into  possible  answers  to  these 
questions,  this  research  effort  adopted  a  three-pronged  approach. 
First,  the  researchers  carried  out  a  comprehensive  review  of  published 
and  unpublished  reports  and  other  studies  on  the  theory  and  imple¬ 
mentation  of  evolutionary  acquisition.  Second,  the  researchers  con¬ 
ducted  a  wide-ranging  series  of  interviews  with  senior  DoD  and  Air 
Force  acquisition  management  officials  regarding  their  understanding 
of  the  meaning  and  implications  of  DoD’s  mandated  EA  policies. 
Finally,  the  researchers  carefully  reviewed  five  major  space  acquisition 
programs  that  have  been  recently  restructured  in  accordance  with  EA 
concepts,  in  order  to  understand  lessons  learned  to  date  on  the  im¬ 
plementation  of  EA.  The  information  on  these  case  studies  was  de¬ 
rived  from  open  sources  and  from  interviews  with  senior  program 
officials.6  The  five  case  studies  reviewed  are  as  follows: 


6  The  interviews  were  conducted  by  the  authors  and  took  place  primarily  during  the  first 
nine  months  of  2004. 
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•  Space-Based  Space  Surveillance  (SBSS)  System 

•  Rapid  Attack  Identification,  Detection,  and  Reporting  System 
(RAIDRS) 

•  Global  Positioning  Satellite  (GPS)  III 

•  Space-Based  Radar  (SBR) 

•  Kinetic  Energy  Interceptor  (KEI) 

Tables  1.1  and  1.2  list  the  organizations  and  the  categories  or 
positions  of  individuals  interviewed  for  this  research  effort.  In  addi¬ 
tion  to  the  interviews  identified  in  these  two  tables,  the  Air  Force  also 
invited  the  researchers  to  vet  initial  research  findings  from  this  re¬ 
search  effort  at  a  closed  forum  of  senior  government  and  industry  of¬ 
ficials  organized  by  the  Space  and  Missile  Systems  Center  (SMC)  Di¬ 
rectorate  of  Contracting  (“Teaming  for  Transformation,”  2004).  This 
forum  included  approximately  100  senior  current  and  former  DoD, 
Air  Force,  and  industry  leaders,  many  of  whom  provided  important 
feedback  for  the  project  research  approach  and  interim  findings. 

Later,  this  chapter  discusses  the  rationale  for  focusing  on  space 
programs  and  for  the  selection  of  these  specific  acquisition  efforts. 

Table  1.1 

Case  Study  Program  Office  Personnel  Interviewed 

Programs  Positions  or  Categories  of  Individuals  Interviewed 


SBSS  Chief,  Acquisition  Management 

RAIDRS  Program  Manager  (SMC/Space  Superiority  Program  Office);  Deputy 
Program  Manager  (SMC/Space  Superiority  Program,  Contracting); 
Acquisition  Developments,  Acquisition  Center  of  Excellence  (SMC) 

GPS  III  Deputy  Program  Manager  (SMC/GPS  III  Program  Office);  Space  Segment 
Branch  Chief;  Contractor  Acquisition  Support  staff  (Science  Applications 
International  Corporation  [SAIC]) 

SBR  Program  Manager  (SMC/SBR  Program  Office);  Chief,  SBR  Business 

Operations;  Deputy  Chief,  SBR  Business  Operations;  Contractor  Acquisition 
Support  staff  (Tecolate  Research,  SAIC,  Aerospace  Corp);  Acquisition 
Developments,  Acquisition  Center  of  Excellence  (SMC) 

KEI  Program  Manager;  KEI  contracting  and  cost  analysis  personnel 
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Table  1.2 

Other  Organizations  and  Individuals  Interviewed 


Position  of  Categories 

Organization  of  Individuals  Interviewed 


SMC  Directorate  of  Contracting 

SMC  Comptroller 

SMC  Development  and  Transformation 
Directorate 

SMC  Acquisition  Center  of  Excellence 
SMC  Directorate  of  Systems  Acquisition 
SMC  Industry  Day 


Deputy  Director,  Contracting 

Chief,  Pricing  (Resource  Mgmt,  Pricing); 

Chair,  Contracting  Committee 

Staff  personnel 

Chief,  Integration  Division 

Staff  personnel 
Staff  personnel 

Industry  and  government  acquisition 
personnel 


Why  This  Project? 

Drawing  from  the  brief  preceding  discussion,  the  basic  rationale  justi¬ 
fying  the  research  reported  in  this  monograph  can  be  summarized  as 
follows:7 

•  Despite  numerous  attempts  at  clarification  by  the  Office  of  the 
Secretary  of  Defense  (OSD),  the  definitions  of  critical  terms  re¬ 
lated  to  EA  and  the  key  practical  elements  of  specific  strategies 
for  implementing  EA  remain  ambiguous  and  lack  clarity,  in  the 
view  of  many  cost  analysts  and  acquisition  managers. 

•  Advocates  and  opponents  of  EA  present  numerous  claimed 
theoretical  benefits  or  likely  challenges  that  will  arise  from  the 
widespread  implementation  of  EA.  Yet  little  systematic  evidence 
has  been  collected  from  actual  programs  to  support  the  claims  of 
either  advocates  or  opponents  of  EA. 


7  This  subsection  is  based  largely  on  the  authors’  distillation  of  key  points  derived  from  nu¬ 
merous  interviews  with  senior  DoD  and  Air  Force  cost  analysts  and  acquisition  managers,  as 
well  as  a  wide  survey  of  the  relevant  published  literature. 
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The  remainder  of  this  subsection  reviews  these  points  in  more 
detail.  It  begins,  however,  with  a  brief  historical  examination  of  the 
motivations  and  objectives  behind  the  formulation  of  EA,  and  how 
its  advocates  originally  perceived  it. 

Dissatisfaction  with  the  Traditional  Acquisition  Process  and  the 
Promise  of  Evolutionary  Acquisition 

For  decades,  observers  have  criticized  the  growing  length  of  time  re¬ 
quired  to  design  and  develop  major  DoD  weapon  systems,  particu¬ 
larly  when  contrasted  with  the  typical  development  times  for  complex 
commercial  products.  This  criticism  became  particularly  strident  in 
the  1990s,  as  the  cycle  times  for  the  development  of  new  generation 
commercial  electronics  continued  to  shrink,  leading  to  situations 
where  electronic  components  and  subsystems  designated  for  use  in 
such  complex  weapon  systems  as  the  F-22  fighter  actually  became 
obsolete  and  disappeared  from  the  marketplace  prior  to  the  comple¬ 
tion  of  system  development.8 

The  traditional  DoD  acquisition  process,  as  laid  out  in  the  DoD 
5000  acquisition  policy  guidance  series,  was  essentially  a  rather  in¬ 
flexible,  serial,  linear,  step-by-step  approach  determined  in  great  de¬ 
tail  by  an  extensive  accumulation  of  DoD  bureaucratic  tradition, 
formal  policy  guidance,  regulations,  and  public  law.  In  addition  to 
the  widely  recognized  problem  of  inordinately  long  development  cy¬ 
cles  that  commonly  spanned  ten  to  fifteen  years,  two  other  key  prob¬ 
lems  particularly  caught  the  attention  of  senior  DoD  officials  in  the 
late  1990s:9 


8  Studies  conducted  by  the  Semiconductor  Industry  Association  and  others  in  the  1990s 
indicated  that  the  average  life  cycle  of  commercial-grade  integrated  circuits  (ICs)  had  de¬ 
clined  to  two  to  five  years,  down  from  five  to  twelve  years  in  the  1980s.  A  large  weapon  sys¬ 
tem  at  this  time  typically  took  around  ten  years  to  develop  and  was  expected  to  remain  in  the 
inventory  for  at  least  30  years.  Thus  several  generations  of  ICs  might  emerge  and  disappear 
during  the  system  design  and  development  phases  alone.  Prior  to  the  first  flight  of  the  F-22 
fighter  in  September  1997,  an  Air  Force  study  concluded  that  hundreds  of  electronic  parts 
planned  for  use  in  the  fighter  had  already  or  would  soon  become  obsolete,  and  would  no 
longer  be  available  (Lorell  et  al.,  2000). 

9  For  a  good  discussion  of  the  perceived  problems  with  the  traditional  acquisition  process 
and  how  they  encouraged  the  formulation  of  EA,  see  Wellman  (2003). 
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1 .  The  traditional  acquisition  process  often  necessitated  the  identifi¬ 
cation  and  detailed  formulation  of  all  the  program  system  per¬ 
formance  requirements  at  the  very  earliest  stage  of  the  program, 
often  before  a  full  understanding  of  the  ultimate  system  perform¬ 
ance  requirements  could  be  attained,  and  when  the  technologies 
to  achieve  the  necessary  capabilities  were  either  immature  or  un¬ 
known.  Nonetheless,  these  requirements,  once  formally  approved, 
would  often  become  cast  in  stone  as  an  unchangeable  template 
against  which  all  aspects  of  the  program  were  judged,  regardless  of 
changing  or  newly  emerging  operational  needs.  This  could  lead  to 
bitter  mutual  recriminations  between  the  acquisition  and  the  op¬ 
erational  user  communities,  as  well  as  criticism  from  Congress,  on 
who  was  responsible  for  fielding  a  system  of  less  than  optimal  ca¬ 
pability.  The  Operational  Test  and  Evaluation  (OT&E)  commu¬ 
nity  would  then  dutifully  and  rigorously  test  each  system  to  en¬ 
sure  it  precisely  met  the  original  formal  written  requirements, 
even  though  those  requirements  might  no  longer  reflect  current 
operational  needs  because  they  were  formulated  so  early  in  the 
program. 

2.  The  traditional  system,  following  the  dictates  of  public  law,  re¬ 
quired  the  formal  development  of  a  detailed  life-cycle  cost  (LCC) 
estimate  during  the  early  developmental  stages  of  the  program. 
These  point  estimates  might  be  required  to  cover  many  years  of 
technology  and  system  development,  as  well  as  procurement,  and 
operations  and  support  periods  of  40  years  or  more.  Not  surpris¬ 
ingly,  these  estimates  often  proved  to  be  unrealistic,  because  so 
many  uncertainties  and  technology  risks  exist  early  in  the  devel¬ 
opmental  cycle.  The  result  was  often  major  cost  overruns,  sched¬ 
ule  slippage,  criticism  from  Congress,  and  mutual  recriminations 
over  who  was  at  fault  among  the  acquisition,  user,  and  contractor 
communities.10 


10  The  authors  are  grateful  to  RAND  colleague  and  former  Air  Force  acquisition  official 
John  C.  Graser  for  his  succinct  formulation  of  these  problems  on  which  they  draw  heavily. 
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By  the  mid-1990s,  the  senior  DoD  leadership  was  seriously  con¬ 
sidering  a  major  overhaul  of  the  entire  DoD  acquisition  process,  in¬ 
cluding  a  major  revision  of  the  DoD  5000  policy  documents,  to  fix 
these  and  other  perceived  procurement  process  problems.  EA  rapidly 
became  a  central  component  in  this  acquisition  reform  effort.  Re¬ 
formers  were  inspired  by  a  variety  of  innovative  acquisition  ap¬ 
proaches  being  tried  outside  the  mainstream  acquisition  process  at 
this  time.  Many  of  the  most  important  concepts  that  lay  behind  the 
EA  concept  emerged  from  early  acquisition  reform  legislation  devel¬ 
oped  in  conjunction  with  Congress  and  the  Defense  Advanced  Re¬ 
search  Projects  Agency  (DARPA)  in  the  early  1990s  to  facilitate  op¬ 
erational  and  hardware  development  of  innovative  new  platforms  and 
technologies,  among  the  most  prominent  of  which  were  large  sur¬ 
veillance  unmanned  aerial  vehicles  (UAVs)  (Leonard  and  Drezner, 
2002). 

The  most  important  acquisition  reform  tools  developed  at  this 
time  included  the  Advanced  Concept  Technology  Demonstration 
(ACTD)  designation,  and  the  use  of  Section  845/804  Other  Transac¬ 
tion  Authority  (OTA).  These  and  other  reform  measures  permitted 
DARPA  to  experiment  simultaneously  with  hardware  and  operational 
concepts  for  development  of  surveillance  UAVs  without  being  bur¬ 
dened  by  the  vast  regulatory  and  statutory  requirements  of  the  stan¬ 
dard  DoD  acquisition  process.  Most  importantly  for  the  future  emer¬ 
gence  of  the  EA  concept,  the  ACTD  approach  did  not  necessitate  a 
complete  and  precise  understanding  of  the  eventual  operational  per¬ 
formance  requirements  for  the  system  at  program  inception,  or  the 
determination  of  detailed  technical  system  requirements  for  the 
hardware.  Rather,  the  ACTD  concept  envisioned  a  highly  flexible 
exploratory  technology  demonstration  process,  which  would  lead  to 
the  unfolding  of  the  precise  operational  utility  and  technological  re¬ 
quirements  for  a  system  through  feedback  from  field  testing  and  ex¬ 
perimentation  with  demonstrator  prototypes. 
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With  the  apparent  early  success  of  DARPA  ACTD  programs 
such  as  the  Global  Hawk  and  DarkStar  UAVs,11  DoD  began  search¬ 
ing  for  ways  to  transfer  some  of  the  key  aspects  of  the  more  flexible 
approach  of  ACTDs  to  the  mainstream  world  of  traditional  acquisi¬ 
tion  programs.  Since  DoD  officials  realized  that  the  whole  traditional 
acquisition  process  could  not,  and  probably  should  not,  be  jettisoned 
in  its  entirety,  the  focus  shifted  to  several  key  elements  of  ACTDs 
and  other  experimental  approaches  that  were  thought  to  be  most 
likely  to  bring  about  beneficial  results. 

On  a  broad  conceptual  level,  two  key  principles  that  were  even¬ 
tually  folded  into  the  emerging  EA  concept  could  be  summarized  as 
follows: 

1 .  The  breaking  down  of  large,  long,  inflexible  traditional  acquisition 
programs  into  several,  much  shorter,  lower  risk,  more  manageable 
phased  increments  or  spirals 

2.  The  transformation  of  the  early  phases  of  the  research  and  devel¬ 
opment  (R&D)  segment  of  acquisition  programs  into  more  flexi¬ 
ble  technology  demonstration  programs  that  would  permit  greater 
experimentation  with  requirements,  operational  concepts,  and 
technology  applications. 

Ambiguity  and  Lack  of  Clarity  in  Definitions  and  Implementation 
Strategies 

By  the  beginning  of  the  new  century,  these  basic  concepts  underlying 
EA  had  gelled  in  the  minds  of  senior  DoD  acquisition  managers  and 
reformers,  eventually  leading  to  the  major  revision  of  the  DoD  5000 
series  promulgated  in  October  2000  and  clarified  in  May  2003.  As 
noted  previously,  the  revised  5000  series  placed  a  central  emphasis  on 
EA  as  the  preferred  acquisition  approach.  The  initial  promulgation  of 
EA,  however,  led  to  considerable  confusion  and  uncertainty  within 


11  Despite  its  promising  beginning,  the  DarkStar  UAV  was  cancelled  in  February  1999  due 
to  design  shortcomings.  While  the  Global  Hawk  is  widely  considered  to  be  an  operational 
success,  the  program  experienced  significant  cost  growth,  particularly  in  2004-2005  during 
development  of  a  heavily  modified  new  version  of  the  UAV. 
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the  traditional  acquisition  community.  This  was  partly  because  many 
of  the  fundamental  concepts  underpinning  DoD’s  EA  strategies  were 
not  new,  and  appeared  to  some  observers  to  be  similar  to  related  con¬ 
cepts  tried  in  the  past.  Confusion  arose  in  some  quarters  regarding 
how  EA  differed  from  the  earlier  concepts.  In  addition,  the  original 
official  formulations  of  EA  were  less  than  perfectly  clear  regarding  the 
definition  of  many  concepts,  and  the  precise  means  of  implementing 
the  policy.  Finally,  many  if  not  most  of  the  old  regulatory  require¬ 
ments,  which  were  usually  waived  in  ACTD  programs  and  other  in¬ 
novative  approaches,  appeared  to  some  observers  to  have  remained 
embedded  in  the  new  revised  5000  series  regulations,  which  seemed 
to  conflict  with  the  basic  goals  of  acquisition  reform. 

As  ably  documented  by  Richard  Sylvester  and  Joseph  Ferrara 
among  others  (Sylvester  and  Ferrara,  2003),  the  adoption  of  strategies 
related  to  EA  has  been  advocated  by  various  DoD  and  other  studies 
for  at  least  twenty  years.  Indeed,  as  far  back  as  the  late  1960s,  some 
DoD  officials  began  urging  adoption  of  a  related  acquisition  strategy 
called  preplanned  product  improvement  (P3I),  in  which  systems 
would  be  designed  from  their  inception  to  incorporate  later  upgrades 
and  newer  technologies  as  they  matured. 12  Phased  acquisition  was 
another  acquisition  reform  measure  with  some  characteristics  similar 
to  EA  that  was  often  advocated  in  the  1970s  and  1980s.13  Another 
related  concept,  spiral  development,  gained  widespread  support  in  the 
computer  software  industry  during  the  1980s.14  Indeed,  according  to 
Sylvester  and  Ferrara,  as  early  as  1984,  the  logistics  commanders  of  all 
three  services  endorsed  evolutionary  acquisition  as  a  legitimate  strat¬ 
egy  and  advocated  development  of  a  formal  policy  guide. 


12  This  includes  unpublished  1981  RAND  research  on  product  improvement  strategies. 

13  For  example,  from  1976  through  1978,  RAND  examined  the  phased  acquisition  strategy 
through  case  study  analysis  of  the  C-5A  military  transport,  the  TF-41  jet  engine,  and  the 
F-lll  fighter  bomber.  These  case  studies  resulted  in  unpublished  internal  reports.  Some  of 
the  results  of  this  work  were  published  in  Lee  (1983). 

14  A  very  influential  article  advocating  an  evolutionary  approach  to  software  development 
was  published  by  Barry  Boehm  (1988).  See  also  Boehm  and  Hansen  (2001). 
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Yet  while  forms  of  evolutionary  acquisition  and  related  concepts 
had  been  discussed  and  advocated  for  years,  prior  to  2000  none  of 
them  were  ever  codified  in  the  5000  series  as  the  preferred  DoD  ap¬ 
proach  to  acquisition.  In  1998,  Jack  Gansler,  then  Under  Secretary  of 
Defense  for  Acquisition  &  Technology,  launched  a  series  of  reviews, 
called  the  Section  912  studies,  aimed  at  reforming  the  overall  DoD 
acquisition  process.  The  following  year,  the  Section  912  study  team 
as  well  as  the  Chairman  of  the  Joint  Chiefs  of  Staff  (JCS)  formally 
endorsed  EA.  Soon  afterward,  Gansler,  who  enthusiastically  sup¬ 
ported  EA,  established  a  dedicated  team  to  revise  the  DoD  5000  ac¬ 
quisition  policy  guidance  series  to  reflect  the  findings  of  the  Section 
912  study.  In  October  2000,  DoD  published  a  major  revision  of  the 
5000  series  Acquisition  System  guidance  regulations  that  mandated 
evolutionary  acquisition  strategies  relying  on  a  spiral  development 
process  as  the  preferred  DoD  acquisition  approach  (DoD,  2000b). 

The  publication  of  the  new  5000  series  regulations  led  to  wide¬ 
spread  uncertainty  and  confusion  among  DoD  acquisition  managers 
regarding  the  terminologies  associated  with  evolutionary  acquisition 
and  the  appropriate  means  for  implementing  the  policy  (Sylvester  and 
Ferrara,  2003;  Slate,  2002;  “Evolutionary  Acquisition  and  Spiral  De¬ 
velopment,”  2002).  Differing  interpretations  of  new  strategies  and 
various  terminologies  soon  proliferated.  The  differences  and  policy 
distinctions,  if  any,  among  concepts  such  as  evolutionary  acquisition, 
phased  acquisition,  incremental  development,  spiral  development, 
and  preplanned  product  improvement,  as  well  as  more  traditional 
concepts  such  as  block  upgrades,  planned  modifications,  and  engi¬ 
neering  change  proposals  (ECPs),  remained  unclear  and  ambiguous 
to  many  DoD  acquisition  managers.15 

This  confusing  situation  led  to  a  lengthy  and  sometimes  difficult 
process  of  attempting  to  clarify  the  meaning  and  practical  implica¬ 
tions  of  the  new  policy  and  how  it  might  be  implemented,  as  docu¬ 
mented  in  part  by  Sylvester  and  Ferrara  (2003).  This  process  began  in 
earnest  with  a  memorandum  issued  on  April  12,  2002,  by  E.  C. 


15  For  example,  see  Burke  (2002). 
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“Pete”  Aldridge,  then  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (USD[AT&Lj)  (Aldridge,  2002a;  also  see 
“Evolutionary  Acquisition  and  Spiral  Development,”  2002).  This 
memorandum  openly  recognized  that  “some  confusion”  persisted  re¬ 
garding  the  definition  of  key  EA  terms,  their  relationship  to  other 
terms,  and  the  implications  of  EA  for  various  aspects  of  the  acquisi¬ 
tion  process  such  as  “contracting  and  requirements  generation.”  In  an 
attempt  to  reduce  confusion  and  ambiguity,  an  attachment  to  the 
memorandum  included  detailed  approved  definitions  of  evolutionary 
acquisition,  spiral  development,  preplanned  product  improvement, 
and  other  concepts. 

The  Aldridge  memorandum  helped  refine  the  EA  concept  fur¬ 
ther,  but  did  not  end  the  confusion  and  uncertainty  surrounding  im¬ 
plementation  of  EA.  A  year  and  a  half  later,  as  part  of  a  wider  effort 
to  overhaul  and  streamline  the  entire  acquisition  system,  Deputy  Sec¬ 
retary  of  Defense  Paul  Wolfowitz  issued  two  new  and  important 
memoranda.  The  first  cancelled  the  revised  Department  of  Defense 
Directive  (DoDD)  5000.1  from  October  2000  and  other  5000  series 
regulations,  and  the  second  provided  interim  guidance  and  directed 
preparation  of  revised  regulations  within  four  months  (Wolfowitz, 
2002).  These  memoranda  were  followed  by  extensive  discussion  and 
debate  among  all  the  major  stakeholders  involved  in  defense  acquisi¬ 
tion.  The  intensity  of  that  debate  and  the  number  of  process  issues 
that  needed  to  be  addressed  delayed  approval  of  the  revised  acquisi¬ 
tion  regulations  for  many  months.  Much  of  the  discussion  revolved 
around  issues  related  to  EA.  Finally,  in  May  2003,  Deputy  Secretary 
Wolfowitz  signed  the  new  DoDD  5000.1  and  Department  of  De¬ 
fense  Instruction  (DoDI)  5000.2  regulations  (DoD,  2003a,  2003c). 

The  new  5000  series  documents  contain  much  more  explicit 
policy  guidance  and  definitions  of  EA  terminology  than  earlier 
documents.  DoDD  5000.1  and  DoDI  5000.2  make  three  fundamen¬ 
tal  points  regarding  EA.  First,  “evolutionary  acquisition  strategies  are 
the  preferred  approach  to  satisfying  operational  needs”  (DoD,  2003a, 
p.  3).  Second,  there  are  two  different  developmental  processes  to  im¬ 
plement  EA  strategies:  incremental  development  and  spiral  develop- 
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ment.  And,  finally,  of  the  two,  “spiral  development  is  the  preferred 
process  for  executing  such  strategies”  (DoD,  2003a,  p.  3). 

Of  critical  importance  of  course  were  the  definitions  finally 
adopted  in  these  documents  for  EA  terminology,  and  particularly 
how  spiral  development  and  incremental  development  relate  to  EA. 
DoDI  5000.2  provides  the  definition  of  EA  quoted  at  greater  length 
at  the  beginning  of  this  chapter,  the  key  phrase  of  which  notes  that 
EA  is  “an  evolutionary  approach”  that  “delivers  capability  in  incre¬ 
ments,  recognizing,  up  front,  the  need  for  future  capability  improve¬ 
ments”  (DoD,  2003c,  p.  4). 

The  incremental  development  process  for  implementing  EA  is 
defined  as  a  process  where  “a  desired  capability  is  identified,  an  end- 
state  requirement  is  known,  and  that  requirement  is  met  over  time  by 
developing  several  increments,  each  dependent  on  available  mature 
technology”  (DoD,  2003c,  p.  5).  However,  the  preferred  process  of 
spiral  development,  as  defined  in  DoDI  5000.2,  suggests  a  much 
more  radical  departure  from  traditional  practice: 

In  this  process  [spiral  development],  a  desired  capability  is  iden¬ 
tified,  but  the  end-state  requirements  are  not  known  at  program 
initiation.  Those  requirements  are  refined  through  demonstra¬ 
tion  and  risk  management;  there  is  continuous  user  feedback-,  and 
each  increment  provides  the  user  the  best  possible  capability. 

The  requirements  for  future  increments  depend  on  feedback  from  us¬ 
ers  and  technology  maturation.  (DoD,  2003c,  p.  5.  Emphasis 
added.) 

Of  the  two  processes  for  implementing  EA,  the  preferred 
approach — spiral  development — is  the  most  challenging.  The  precise 
nature  of  the  spiral  development  process  as  applied  to  the  develop¬ 
ment  of  weapon  system  hardware  was  not  spelled  out  in  detail  in  the 
DoD  guidance  documents,  and  remained  an  area  of  some  contention 
and  debate.  As  touched  on  above,  the  spiral  development  concept 
originated  some  years  earlier  from  the  commercial  and  academic 
software  development  worlds,  but  was  not  originally  thought  of  as 
being  applicable  to  hardware  development.  Dr.  Barry  Boehm  is 
widely  credited  with  developing  and  popularizing  the  spiral  develop- 
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ment  process  while  Chief  Scientist  at  TRW,  Inc.,  in  the  mid-1980s  as 
a  means  of  reducing  risk  and  increasing  responsiveness  to  customer 
needs  in  large  commercial  software  development  projects  (“What  Is 
Spiral  Development?”  2003;  see  also  Boehm,  2000).  A  key  element  of 
the  original  spiral  concept  was  the  progressive  serial  development  of 
numerous  software  prototypes  during  each  developmental  spiral  or 
increment;  these  prototypes  would  be  tested  by  users,  and  the  opera¬ 
tional  insights  gained  would  then  be  funneled  back  through  a  formal 
feedback  loop  into  the  requirements  and  development  processes  for 
the  next  spiral. 

Most  early  advocates  of  spiral  development  in  the  weapon  sys¬ 
tem  acquisition  arena  emphasized  a  similar  concept  of  spiral  proto¬ 
type  development  with  feedback  loops  (Little,  2002).  The  precise  na¬ 
ture  and  extent  of  the  feedback  loop  process  in  large  defense 
equipment  programs  eventually  became  an  area  of  contention.  Most 
observers  came  to  view  the  feedback  loop  process  for  hardware  (as 
opposed  to  software)  as  only  taking  place  as  a  subprocess  to  help  refine 
requirements  within  each  increment,  each  of  which  would  be  already 
fairly  well  defined  at  the  beginning  of  the  increment  (see,  for  exam¬ 
ple,  McNutt,  2000;  and  Brown,  2003).  The  precise  meaning  and  ex¬ 
tent  of  the  feedback  loop  process  remains  vague  and  ill  defined  in  the 
most  recent  DoD  5000  series  guidance.  As  quoted  above,  DoDI 
5000.2  clearly  states  that  in  spiral  development  “(t)he  requirements 
for  future  increments  depend  on  feedback  from  users  and  technology 
maturation”  (DoD,  2003c,  p.  5)  But  it  also  states  that  “  there  is  con¬ 
tinuous  user  feedback ”  (DoD,  2003c,  p.  5)  during  spiral  development, 
implying  that  it  takes  place  both  within  spirals  and  between  spirals. 
The  questions  of  field  testing  of  progressive  prototypes  and  imple¬ 
menting  feedback  loops  from  users  into  the  R&D  process  during  spe¬ 
cific  spirals  or  increments  later  raised  many  perceived  challenges  for 
program  managers,  particularly  regarding  the  issue  of  requirements 
creep,  as  touched  on  further  in  Chapter  Two. 

Thus,  EA  implemented  through  spiral  development,  as  defined 
in  the  May  2003  revision  of  the  5000  series  documents,  could  be  and 
was  interpreted  as  representing  a  potentially  radical  departure  from 
traditional  DoD  acquisition  policy.  While  EA  entails  a  phased,  in- 
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cremental  approach  to  achieving  a  capability,  the  mandated  prefer¬ 
ence  for  the  use  of  the  spiral  development  process  to  implement  this 
strategy  indicates  that  the  end-state  system  requirements  are  not 
known  at  program  inception,  but  rather  evolve  throughout  develop¬ 
ment  through  an  iterative  process.  Older,  related  concepts,  such  as 
P3I,  differ  significantly  from  EA  using  spiral  development  in  that 
with  P3I  both  the  end-state  capabilities  and  system  requirements  are 
known  at  program  inception.  This  distinction  later  proved  to  be  of 
crucial  importance  for  cost  analysts  and  for  budgeters,  as  discussed 
further  in  Chapter  Two. 

Another  critical  distinction  between  EA — using  either  the  spiral 
development  or  incremental  development  process — and  older,  related 
concepts,  such  as  P3I,  is  that  with  EA  the  specific  number,  makeup, 
content,  and  objectives  of  the  various  upgrade  increments  throughout 
the  program  life  cycle  are  largely  unknown  at  program  inception.  In 
the  case  of  P3I,  the  specific  upgrades  and  their  timing  and  phasing 
are  well  known  in  advance. 16 

Thus,  in  spite  of  the  clarifications  provided  by  the  revised  DoD 
5000  series  in  May  2003,  the  formal  adoption  of  EA  using  the  pre¬ 
ferred  approach  of  spiral  development  or  incremental  development 
raised  many  major  questions  for  acquisition  program  managers  re¬ 
garding  the  implementation  of  life-cycle  cost  estimating,  budgeting, 
system  requirements  formulation,  and  other  key  program  processes, 
some  of  which  are  discussed  in  greater  detail  below.  As  Sylvester  and 
Ferrara  so  rightly  noted  in  2003, 

Because  EA  has  never  been  implemented  in  a  wholesale  fashion 
within  the  DoD,  no  one  is  exactly  sure  of  how  its  implementa¬ 
tion  will  play  out,  but  everyone  is  pretty  sure  that  full  implemen¬ 
tation  of  EA,  as  called  for  in  the  5000  series,  will  probably  mean 


16  The  authors  are  unaware  of  any  definitive,  official  DoD  documents  that  clearly  and  con¬ 
sistently  distinguish  between  EA,  P3I,  block  upgrades,  and  other  historical  concepts  with 
similarities  to  EA.  One  quasiofficial  source  makes  the  following  distinction  regarding  EA  and 
P3I,  which  is  slightly  different  from  what  is  shown  above:  Unlike  EA,  this  source  claims  that 
“pre-planned  product  improvement  (P3I)  is  a  traditional  acquisition  strategy  that  provides 
for  adding  improved  capability  to  a  mature  system”  (“Evolutionary  Acquisition  and  Spiral 
Development,”  2002). 
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major  changes  in  the  way  DoD  has  traditionally  done  business. 
(Sylvester  and  Ferrara,  2003,  p.  10.  Emphasis  added) 


Theoretical  Benefits  and  Challenges:  What  Did  the  Reformers  Intend 
EA  to  Be,  and  What  Process  Did  the  New  5000  Series  Documents 
Establish? 

Claimed  Benefits.  The  advocates  of  EA  using  spiral  development 
argue  that  the  benefits  that  are  likely  to  accrue  justify  accepting  the 
difficult  challenges  that  may  be  required  to  implement  the  policy.  As 
already  noted,  the  most  important  benefit  claimed  for  EA  was  more 
rapid  fielding  of  operationally  useful  capability  than  the  old  single- 
step  to-full-capability  approach.  Other  claimed  benefits  include  the 
following: 

•  Reduced  likelihood  of  major  R&D  schedule  delays  and  cost 
overruns  due  to  the  focus  on  the  use  of  mature  technologies  and 
the  promotion  of  more  realistic  system  performance  expecta¬ 
tions 

•  Enhanced  requirements  formulation  process,  which  is  much 
more  responsive  to  the  warfighter’s  real  operational  needs  due  to 
constant  feedback  from  the  user  communities 

•  Rapid  insertion  of  the  latest  technologies  into  the  system,  thus 
avoiding  the  problem  of  obsolete  parts  and  subsystems. 

As  noted  earlier,  the  key  to  achieving  these  benefits  through  EA 
is  the  separation  of  large,  complex  programs  that  envision  a  “single 
step  to  capability”  into  many  smaller,  more  manageable  phased  steps 
or  increments.  Figure  1.1  contrasts  a  highly  simplified  theoretical 
graphic  comparison  of  the  traditional  approach  to  the  EA  approach. 
As  shown  in  the  top  part  of  the  figure,  the  traditional  process  is  a  sin- 
gle-step-to-capability  approach,  wherein  a  single,  very  long,  high-risk 
R&D  and  procurement  effort  is  undertaken  aimed  at  achieving  a 
rigid  set  of  very  demanding  capabilities  based  on  technologies  which 
may  be  uncertain  or  immature  at  the  beginning  of  the  program.  In 
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Figure  1.1 

Single  Step  Versus  Evolutionary  Acquisition  Approach 


Time 


SOURCE:  Lumb  (2004,  p.  12). 

NOTES:  IOC  =  initial  operational  capability;  FOC  =  full  operational  capability. 
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the  EA  approach,  shown  on  the  bottom  of  Figure  1.1,  such  a  pro¬ 
gram  is  broken  up  into  many  smaller,  shorter,  more  manageable  in¬ 
crements  or  spirals. 

Each  EA  program  increment  is  treated  in  theory  as  if  it  is  a  sepa¬ 
rate  program  with  its  own  unique  developmental  and  procurement 
phases. 

A  series  of  less  demanding,  phased  capability  steps  are  identified 
that  more  closely  match  the  likely  maturity  of  the  component  tech¬ 
nologies  at  any  given  time.  Thus  the  available  technologies  are 
matched  with  more  realistic  requirements  and  capability  expectations 
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over  time,  resulting  in  multiple  steps  of  increasing  capability  made 
available  much  sooner  to  the  warfighter.  In  principle,  each  step,  while 
not  resulting  in  the  ultimate  objective  capability,  provides  the  war¬ 
fighter  with  an  improved  capability  that  is  superior  to  the  existing 
capability  in  the  field  and  that  is  operationally  useful.  Ultimately,  the 
same  high  objective  capability  is  attained,  but  not  before  the  war¬ 
fighter  has  received  many  earlier  variants  with  an  interim  capability 
that  exceeds  existing  fielded  equipment. 

The  building  blocks  or  steps  that  make  up  an  EA  program  are 
actually  viewed  as  a  series  of  multiple,  staggered,  simultaneous  devel¬ 
opment  iterations,  increments,  or  spirals.  Each  of  these  iterations  is, 
for  all  practical  purposes,  a  minisystem  development  program,  which 
includes  most  of  the  standard  phases  and  regulatory  and  oversight 
requirements  laid  out  by  DoDI  5000.2  for  overall  system  acquisition 
programs.  These  iterations  build  on  the  core  increment  to  produce 
improved  variants  or  upgrades  of  the  system  that  emerge  from  the 
core  increment.  This  concept  is  graphically  illustrated  in  Figure  1.2. 17 

The  second  key  aspect  of  EA  is  made  up  of  three  elements.  The 
original  advocates  of  EA  insisted  that  no  system  variant  being  devel¬ 
oped  in  a  given  increment  should  be  permitted  to  enter  into  full-scale 
development  (that  is,  formal  acquisition  program  initiation)  unless  it 
is  determined  that  (1)  the  technologies  are  mature,  (2)  the  require¬ 
ments  are  well-defined,  and  (3)  the  funding  has  been  programmed. 


17  As  is  discussed  in  greater  detail  later  in  this  monograph,  the  new  revised  EA  process  actu¬ 
ally  leads  to  a  substantial  increase  in  the  process  complexity  of  programs.  For  example,  each 
increment  is  required  to  have  all  the  supporting  documentation  for  the  following  categories  of 
factors:  (1)  approved  operational  requirements;  (2)  performance,  cost,  and  schedule  goals; 
(3)  operational  and  live  fire  testing  (if  required);  (4)  compliance  with  all  acquisition  oversight 
requirements;  (5)  a  formal  acquisition  strategy  plan  that  includes  logistics  planning,  man¬ 
power,  personnel  and  training,  environmental  and  security  factors,  protection  of  critical 
program  information,  and  spectrum  management;  and  (6)  other  information  tailored  to  the 
unique  circumstances  of  the  program.  For  details,  see  DoD  (2002).  Also  see  Defense  Acqui¬ 
sition  University  (2003). 
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Figure  1.2 

EA  Implies  Multiple,  Staggered,  Simultaneous  Iterations 
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To  achieve  this  objective,  EA  advocates  sought  to  transform  the 
earliest  phases  of  each  EA  program  increment  or  spiral  into  a  much 
more  flexible  technology  and  concept  demonstration  phase.  The 
main  purposes  were  to  ensure  that  available  technologies  had  reached 
adequate  levels  of  maturity,  and  that  the  system  requirements  and 
cost  estimates  had  been  sufficiently  refined.  Strict  entrance  criteria  for 
moving  into  the  next  phase — now  called  system  development  and 
demonstration  (SDD) — would  be  required,  since  this  phase  would 
constitute  acquisition  program  initiation.  The  goal  was  to  promote 
the  exclusive  use  of  sufficiently  mature  technologies  to  avoid  high  de¬ 
velopmental  risk,  the  achievement  of  stable  system  requirements,  and 
the  provision  of  full  funding  for  the  entire  increment  or  spiral.  Pro¬ 
gram  initiation  (Milestone  B)  would  not  be  approved  until  these  cri¬ 
teria  were  met.  These  key  aspects  of  the  EA  process  are  illustrated  in 
Figure  1.3,  which  shows  the  overall  technology  development  phase 
after  Milestone  A,  and  discrete  technology  demonstration  phases  pre¬ 
ceding  each  Milestone  B  decision  for  each  separate  increment. 
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Figure  1.3 

Overall  EA  Requirements  and  Acquisition  Process  Depiction 
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SOURCE:  U.S.  Department  of  Defense  (2003c,  Figure  2).  Also  see  Defense  Acquisition 
University  (2003). 

NOTES:  NSS  =  National  Space  Strategy;  NMS  =  National  Military  Strategy;  DOTMLPF  = 
doctrine,  organization,  training,  materiel,  leadership  and  education,  personnel  and 
facilities;  ICD  =  initial  capabilities  document;  CD  =  concept  decision;  AoA  =  analysis  of 
alternatives;  CDD  =  capability  development  document;  CPD  =  capability  production 
document;  DAB  =  Defense  Acquisition  Board;  JROC  =  Joint  Requirements  Oversight 
Council. 
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Figure  1.4  shows  how  the  structuring  of  the  overall  acquisition 
program,  as  well  as  each  discrete  EA  increment,  were  changed  by  the 
revised  5000  series  in  order  to  further  implement  these  policies.  The 
major  changes  were  the  old  concept  and  technology  development 
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ph  ase  was  divided  into  two  separate  phases  (concept  refinement  and 
technology  development);  Milestone  A  was  moved  back  to  the  end  of 
the  concept  refinement  phase  and  before  entrance  into  the  technol¬ 
ogy  development  phase;  the  old  decision  review  assessment  activity 
was  moved  forward  to  the  beginning  of  the  new  concept  refinement 
ph  ase  and  renamed  the  concept  decision  assessment  activity;  and  the 
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Main  Differences  Between  the  Old  and  New  DoDI  5000.2  Acquisition 
Management  Frameworks 
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old  interim  progress  review  assessment  activity  became  the  design 
readiness  review.  The  intent  was  in  essence  to  add  an  additional  for¬ 
mal  milestone  (the  concept  decision  point  was  similar  to  the  old 
Milestone  0),  and  focus  the  Milestone  A  decision  on  planning  for  the 
technology  development  phase.  The  purpose  was  to  promote  greater 
emphasis  on  requirements  definition  and  technology  maturation  ef¬ 
forts  prior  to  program  initiation,  both  for  the  overall  program  and  for 
each  individual  EA  increment. 

The  exit  criteria  from  the  technology  development  phase  are 
characterized  as  the  successful  identification  of  an  “  [a]  ffordable  in¬ 
crement  of  militarily-useful  capability  with  technology  demonstrated 
in  a  relevant  environment,  and  a  system  that  can  be  developed  for 
production  within  a  short  timeframe”  (Lumb,  2004,  p.  19).  Thus, 
the  intent  was  to  not  pass  through  Milestone  B  (formal  program  ini¬ 
tiation)  nor  enter  into  full-scale  development  (system  development 
and  demonstration,  or  SDD)  until  a  clearly  understood  military  ca¬ 
pability  goal  had  been  identified  and  a  relatively  low-risk  technologi¬ 
cal  solution  to  provide  the  capability  was  available.  As  the  authors’ 
space  case  studies  show,  these  objectives  proved  difficult  to  satisfy  on 
many  EA  programs,  however. 

Thus  EA  aims  at  harmonizing  and  synchronizing  the  evolution 
of  military  requirements  and  capability  objectives  with  the  evolution 
and  maturation  of  technology.  This  leads,  according  to  EA  advocates, 
to  the  much  more  rapid  fielding  of  operationally  useful  systems 
through  developmental  programs  that  are  much  less  prone  to  cost 
growth  and  schedule  slippage  because  they  depend  on  mature  tech¬ 
nologies  and  more  realistic  capability  objectives. 

EA  supporters  claim  that  implementing  EA  through  the  spiral 
development  process  has  the  added  advantage  of  producing  systems 
that  are  much  more  responsive  to  the  warfighters’  and  logisticians’ 
needs.  This  is  because  spiral  development,  as  noted  above,  envisions 
multiple  feedback  loops  during  every  development  increment 
wherein  warfighters,  logisticians,  and  testers  can  constantly  provide 
inputs  and  corrections  into  the  development  process  based  on  field 
experience,  using  prototypes  or  technology  demonstrators.  Thus  the 
requirements  and  even  the  capabilities  of  the  system  constantly  evolve 
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in  response  to  real  world  experience.  One  typical  Air  Force  stylized 
graphical  representation  of  this  collaborative  approach  is  shown  in 
Figure  1.5. 18 

Figure  1.5 

Feedback  Loops  Refine  Requirements  in  Spirals:  Notional  Unmanned 
Combat  Aerial  Vehicles  (UCAVs)  as  USAF  Spiral  System  R&D  Illustration 
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18  On  EA’s  enhanced  responsiveness  to  users,  see  Brown  (2002). 
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In  a  traditional  program,  the  major  generic  program  phases 
(concept  development,  development,  preproduction,  and  production) 
are  separate  and  distinct,  with  little  cross  interaction,  as  shown  in  the 
top  half  of  Figure  1.5.  This  contrasts  with  a  spiral  approach,  where  an 
integrated  acquisition  team  made  up  of  users,  acquirers,  testers,  re¬ 
searchers,  and  logisticians  work  together  on  numerous  phased  but 
overlapping  development  cycles  that  are  linked  by  information  feed¬ 
back  loops.  Thus  the  overall  system  requirements  evolve  over  time 
based  on  user  and  logistician  experience,  the  progress  of  technology 
development,  and  so  forth.  As  one  senior  Air  Force  official  put  it,  in 
spiral  development  “the  requirements  evolve  from  user  learning”  (Lit¬ 
tle,  2002,  p.  8). 

The  graphic  overall  portrayal  of  the  constant  requirements  feed¬ 
back  loop  from  the  initial  requirements  generation  through  each 
milestone  review  of  each  increment  is  also  portrayed  in  DoDI 
5000.2,  as  shown  in  Figure  1.3. 

Thus,  the  DoD  5000  regulations  as  revised  in  May  2003  con¬ 
tained  significant  process  and  structural  changes  to  the  overall  theo¬ 
retical  acquisition  process.  To  summarize,  these  were  expected  to  be 
the  key  benefits  of  EA  that  would  flow  from  these  changes: 

•  Much  faster  fielding  of  operationally  useful  capability  to  the 
warfighter 

•  Lower  risk  of  program  cost  growth  or  schedule  slip  due  to 
phased  insertion  of  more  mature  technology 

•  More  responsive  requirements  process  that  leads  to  more 
combat-effective  and  -supportable  systems 

•  More  rapid  insertion  of  new  technologies. 

Potential  Challenges  for  the  Cost  Analysis  Community.  Despite 
the  attractive  benefits  claimed  for  EA,  even  its  most  fervent  support¬ 
ers  recognize  that  it  poses  several  potential  implementation  challenges 
for  the  cost  estimating  and  acquisition  management  community.  For 
example,  a  typical  recent  DoD  briefing  advocating  EA  made  the  key 
point  that  “[w]hile  EA  should  get  systems  to  the  field  faster,  there  are 
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some  complex  areas  that  need  attention”  (Defense  Acquisition  Uni¬ 
versity,  undated,  p.  25). 

Probably  the  most  commonly  mentioned  challenges  facing  the 
cost  analysis  community  are  as  follows: 

•  Costing  spiral  acquisition  programs  (or  increments)  where  the 
end-state  requirement  is  not  known  at  program  inception  (by 
definition),  and  which  continuously  evolves  throughout  pro¬ 
gram  life 

•  Access  to  independently  assessed  technical  and  programmatic  in¬ 
formation  for  each  of  the  spirals. 

Since  the  initial  adoption  of  EA  in  the  revised  DoD  5000  regu¬ 
lations  issued  in  October  2000,  numerous  acquisition  experts  have 
raised  concerns  regarding  the  challenges  posed  by  spiral  development 
to  DoD  Comptroller  functions  such  as  those  related  to  budgeting, 
financing,  contracting,  and  cost  estimating  (see  Sylvester  and  Ferrara, 
2003).  Theoretically  and  by  definition,  neither  the  acquisition  man¬ 
agers  nor  the  ultimate  users  know  what  the  final  or  objective  system 
requirement  is  at  the  beginning  of  a  spiral  development  program.  Yet 
most  of  the  traditional  statutory  and  regulatory  imperatives  dictating 
cost  estimating  and  budgeting  requirements  are  still  applicable.  At  the 
beginning  of  a  program,  how  can  the  cost  of  a  system  and  its  overall 
development  program  be  estimated,  much  less  its  total  expected  life- 
cycle  costs  (LCC),  when  no  one  knows  what  the  objective  require¬ 
ment  for  that  system  will  end  up  being,  or  what  that  system  will  look 
like?  In  the  minds  of  many  observers,  this  remains  the  central  chal¬ 
lenge  posed  by  EA  for  cost  analysts,  and  may  point  to  the  need  for 
EA  programs  to  maintain  larger  cost  reserves  in  their  budgets  to  ad¬ 
dress  the  “known  unknowns”  (for  example,  see  Brown,  2003). 

Expert  observers  also  identified  a  variety  of  other  challenges  for 
cost  analysts  posed  by  EA,  which  are  related  to  the  overlapping 
phased  increments  that  characterize  EA,  early  on.  These  challenges 
involved  the  complexities  of  costing,  contracting,  and  managing  EA 
programs  made  up  of  multiple,  simultaneous,  staggered  increments  or 
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subprograms  for  different  variants  in  different  stages  of  R&D,  pro¬ 
duction,  operation,  and  support. 

As  illustrated  in  Figures  1.2,  1.3,  and  1.5,  evolutionary 
acquisition — whether  using  the  spiral  development  or  incremental 
development  process — resembles  a  collection  of  parallel  overlapping 
but  closely  interconnected  subprograms,  each  in  a  different  phase  of 
the  formal  acquisition  process.  Such  a  program  structure  obviously 
entails  additional  management  complexities  compared  to  a  traditional 
single  increment  program.  Each  increment  or  subprogram  of  an  EA 
program  has  its  own  formal  milestone  and  other  program  reviews, 
requiring  the  generation  of  all  the  usual  formal  documentation  in¬ 
cluding  that  for  costs.  Thus,  some  observers  believed  that  an  EA  pro¬ 
gram  would  greatly  increase  overall  management  and  oversight 
workload  for  program  offices.  Instead  of  managing  one  large  pro¬ 
gram,  under  the  traditional  approach,  a  program  office  might  face  the 
prospect  of  managing  numerous  complex  subprograms  all  in  different 
ph  ases  of  the  acquisition  process,  and  going  through  different  mile¬ 
stones  requiring  different  types  of  oversight  and  reporting. 

Some  of  the  hypothetical  challenges  potentially  raised  by  the 
unique  structuring  of  EA  programs  with  numerous  overlapping  in¬ 
crements  are  summarized  as  follows: 

•  Anticipating  and  costing  future  retrofit  programs  that  may  be 
deemed  necessary  to  upgrade  earlier  variants  with  technology 
developed  in  later  increments,  spirals,  or  iterations 

•  Estimating  increased  indirect  costs  from  the  need  for  greater 
process  and  up-front  management  investments  in  the  early 
stages  of  the  program 

•  Costing  increased  workload  required  to  prepare  for  the  larger 
number  of  milestone  and  submilestone  reviews  necessary  for 
multiple  simultaneous  iterations 

•  Determining  the  effects  on  cost  improvement  curves  of  multiple 
simultaneous  configurations  in  different  stages  of  R&D  and 
production,  plus  possible  simultaneous  retrofit  upgrade  pro¬ 
grams 
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•  Estimating  operations  and  support  (O&S)  costs  for  a  force  made 
up  of  numerous  different  configurations  in  different  stages  of 
the  acquisition  process. 

Potential  Challenges  for  the  Acquisition  Management  Commu¬ 
nity.  In  addition  to  these  more  tactical  concerns  relevant  to  cost  ana¬ 
lysts,  there  are  numerous  other  concerns  identified  by  various  observ¬ 
ers  and  critics  related  to  broader  strategic  challenges  posed  by  EA 
related  to  the  effective  management  of  large  weapon  system  acquisi¬ 
tion  programs.  Some  of  the  most  common  concerns  of  this  type  men¬ 
tioned  by  critics  include  the  following: 19 

•  With  the  ultimate  requirements  unknown  at  program  inception, 
EA  may  encourage  DoD  to  commit  to  early  iterations  of  a  pro¬ 
gram  with  little  insight  into  the  ultimate  cost  and  technical  risk 
entailed  in  reaching  the  full  system  capabilities.20 

•  Contractors  may  allocate  the  relatively  easy  developmental  tasks 
to  early  iterations  of  the  program,  slipping  the  more  difficult 
tasks  to  later  iterations,  in  the  hopes  that  DoD  will  become  ir¬ 
revocably  committed  to  the  program. 

•  Program  offices  and  contractors  may  be  tempted  to  mask  trou¬ 
bled  programs  that  are  failing  to  meet  requirements  by  re¬ 
labeling  them  as  EA  programs. 

•  The  existing  temptation  to  rob  funding  intended  for  later  itera¬ 
tions  to  pay  for  funding  shortfalls  or  cost  overruns  in  current  it¬ 
erations  will  be  increased. 


19  Many  of  these  as  well  as  other  theoretical  concerns  regarding  the  implementation  of  EA 
are  identified  in  Sylvester  and  Ferrara  (2003),  as  well  as  elsewhere. 

20  According  to  the  revised  5000  series  documents,  in  a  spiral  development  program,  the  end 
capabilities  are  known,  but  the  requirement  is  not.  What  this  apparently  means  is  that  the 
desired  end  capabilities  are  known  (for  example,  such  as  tracking  tactical  moving  targets 
from  space  over  an  entire  theater  of  operations),  but  the  precise  technologies  and  technical 
means  of  achieving  those  capabilities  are  not  known  (for  example,  the  precise  numbers  of 
space  vehicles  in  the  constellation,  type  of  radar  technology  and  antenna  array  to  be  used  on 
the  space  vehicles,  or  the  architecture  of  the  communication  links). 
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•  The  user  community  as  well  as  the  developmental  and  opera¬ 
tional  test  and  evaluation  (D/OT&E)  communities  may  have 
difficulty  formally  determining  when  the  end  requirement  has 
been  achieved  in  a  spiral  development  program.21 

•  The  lack  of  clearly  defined  end  requirements  at  program  incep¬ 
tion  may  contribute  to  frequent  changes  in  requirements  as  well 
as  requirements  creep  throughout  development,  factors  that  his¬ 
torically  have  been  identified  as  contributing  to  cost  growth  and 
schedule  slippage. 

There  is  little  doubt  that  the  full  adoption  of  EA  poses  many  po¬ 
tential  implementation  challenges  for  cost  analysts  and  acquisition 
managers.  Yet  practical  experience  with  implementing  EA  remains 
sparse,  so  that  the  various  views  on  the  severity  of  the  challenges 
posed  by  EA  implementation  versus  the  value  of  the  potential  bene¬ 
fits  to  be  derived  from  EA  are  often  still  a  matter  of  speculation  or 
theory.  Little  analysis  to  date  has  been  conducted  on  what  little  prac¬ 
tical  experience  has  been  accumulated. 

In  short,  no  one  presently  knows  with  any  certainty  what  the 
impact  of  full  EA  implementation  will  be  over  the  long  term  on  the 
acquisition  process  and  acquisition  program  outcomes.  No  detailed 
DoD  guidance  of  which  the  authors  are  aware  exists  to  assist  the  cost 
analysis  and  acquisition  management  community  in  implementing 
the  details  of  programs  structured  in  accordance  with  the  precepts  of 
EA.  This  report  attempts  to  review  systematically  the  practical  experi¬ 
ence  with  EA  that  does  exist  in  a  small  group  of  specific  programs  to 
help  shed  greater  light  on  these  issues  and  move  toward  a  better  un¬ 
derstanding  of  how  acquisition  management  and  cost  analysts  are 
contributing  to  the  implementation  of  EA  in  the  real  world. 


21  According  to  the  revised  DoD  5000  documents,  each  increment  must  pass  through  a 
formal  test  and  evaluation  phase  just  like  a  traditional  system.  However,  in  some  nonspace 
EA  programs  such  as  the  Global  Hawk  and  Predator  unmanned  aerial  vehicles  (UAVs),  pro¬ 
totypes  have  been  shipped  to  active  operational  theaters  for  combat  testing  and  use  prior  to 
formal  operational  testing. 
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Research  Approach  and  Organization  of  This  Monograph 

As  noted  in  the  introduction  to  this  monograph  previously,  this  re¬ 
search  effort  adopted  a  three-pronged  approach.  First,  the  authors 
conducted  a  comprehensive  review  of  published  and  unpublished  re¬ 
ports  and  other  studies  on  the  theory  and  implementation  of  evolu¬ 
tionary  acquisition.  Second,  the  authors  systematically  interviewed  a 
variety  of  senior  DoD  and  Air  Force  officials  from  the  cost  analysis 
and  acquisition  management  community  regarding  their  under¬ 
standing  of  the  meaning  and  implications  EA  policies  as  spelled  out 
in  5000  series  regulations  promulgated  in  May  2003.  Finally,  the 
authors  carefully  reviewed  five  major  space  acquisition  programs  that 
have  been  recently  restructured  in  accordance  with  the  new  space  sys¬ 
tem  acquisition  guidance  (see  below  for  specific  cases),  in  order  to 
understand  lessons  learned  to  date  on  the  implementation  of  EA  on 
military  space  programs. 

Why  Space  Programs? 

Early  on,  the  authors  decided  to  select  all  the  major  case  studies  from 
a  specific,  focused  area:  military  space  programs.  There  are  two  basic 
reasons  for  this  decision:  the  growing  importance  of  space  for  DoD 
and  the  Air  Force  and  the  recent  extensive  and  very  detailed  overhaul 
of  DoD  space  acquisition  policy  that  designated  EA  as  the  preferred 
acquisition  approach.  The  rationale  for  selecting  space  programs  as 
case  studies  is  discussed  in  more  detail  below. 

As  mentioned  previously,  the  first  reason  for  focusing  on 
space  acquisition  is  that  the  relative  importance  of  space  pro¬ 
grams  for  OSD  and  the  Air  Force  as  key  transformational  systems 
has  recently  and  rapidly  grown.  Over  the  last  several  years,  the  U.S. 
Air  Force  has  invested  on  the  order  of  $6  billion  a  year  in  unclassified 
space  programs.  In  January  2001,  a  special  high-profile  space  com¬ 
mission  mandated  by  Congress  and  chaired  by  Donald  Rumsfeld 
stressed  the  growing  importance  of  space  to  U.S.  national  security. 
The  space  commission  formally  recommended  that  the  President  re¬ 
view  and  revise  U.S.  national  space  policy  to  better  promote  the  em¬ 
ployment  of  space  assets  “to  help  speed  the  transformation  of  the 
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U.S.  military”  (Commission  to  Assess  United  States  National  Secu¬ 
rity  Space  Management  and  Organization,  2001,  p.  7).  Two  years 
later,  a  key  follow-on  study,  the  so-called  Young  Panel  Report,  con¬ 
cluded  that  “U.S.  national  security  is  critically  dependent  upon  space 
capabilities  and  that  dependence  will  continue  to  grow”  (DoD, 
2003b,  p.  1).  This  increasing  emphasis  on  space  continues  to  be  re¬ 
flected  in  DoD’s  fiscal  year  (FY)  2006  budget  request  to  Congress.  As 
one  observer  noted,  reductions  in  traditional  Air  Force  and  Navy 
programs  “are  at  the  core  of  Defense  Secretary  Donald  Rumsfeld’s 
drive  to  transform  the  military  into  a  more  flexible  and  integrated 
force,  with  space  initiatives  slated  to  receive  a  steadily  growing  slice  of 
the  defense  pie”  (Pasztor  and  Karp,  2005).  While  the  Air  Force  may 
gradually  phase  out  up  to  25  percent  of  its  existing  fighter  aircraft 
force  structure,  spending  on  Air  Force  space  programs  since  2002  has 
risen  by  nearly  $2  billion,  or  about  20  percent. 

As  a  reflection  of  the  growing  importance  of  space,  and  in  re¬ 
sponse  to  the  recommendations  of  the  Space  Commission,  Secretary 
Rumsfeld  directed  OSD  and  the  Air  Force  to  consolidate  authorities 
across  the  national  security  space  community.  As  a  result,  in  2002, 
the  planning,  programming,  and  acquisition  of  all  national  security 
space  programs  were  concentrated  into  the  hands  of  the  Under  Secre¬ 
tary  of  the  Air  Force,  thus  greatly  increasing  the  importance  of  the 
role  played  by  the  Air  Force  in  major  space  acquisition  programs.22 

The  second  rationale  for  focusing  on  space  in  this  study  is 
the  extensive  and  detailed  effort  recently  undertaken  by  the  Air 
Force  to  restructure  space  acquisition  policy  guidance,  with  an 
emphasis  on  FA.  In  2003,  the  Air  Force  launched  an  ambitious  and 
detailed  effort  to  overhaul  the  entire  national  space  acquisition  sys¬ 
tem,  which  was  now  concentrated  for  the  first  time  under  its  direct 


22  In  early  2002,  the  Under  Secretary  of  the  Air  Force,  Peter  B.  Teets,  was  delegated  author¬ 
ity  as  the  Air  Force  Acquisition  Executive  for  Space  for  Air  Force  space  programs  and  DoD 
Milestone  Decision  Authority  for  all  DoD  space  Major  Defense  Acquisition  Programs 
(MDAPs).  In  July  of  that  year,  Teets  became  the  DoD  Executive  Agent  for  Space.  In  addi¬ 
tion,  Teets  is  dual-hatted  as  the  Director  of  the  National  Reconnaissance  Office  (NRO). 
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supervision.  This  overhaul  included  the  formal  incorporation  of  evo¬ 
lutionary  acquisition  as  the  preferred  acquisition  approach. 

The  main  objective  of  the  Young  Panel,  as  mentioned  previ¬ 
ously,  was  to  develop  a  strategy  for  overhauling  and  reforming  the 
national  space  acquisition  system,  in  response  to  the  space  commis¬ 
sion  findings  and  the  concentration  of  space  acquisition  authority  in 
the  hands  of  the  Air  Force.  The  Young  Panel  concluded  that  the 
space  acquisition  system  had  become  profoundly  dysfunctional  and 
was  in  dire  need  of  reform:  “The  Government  capabilities  to  lead  and 
manage  the  [space]  acquisition  process  have  seriously  eroded”  (Office 
of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics,  2003,  p.  3).  Furthermore,  the  Young  Panel  concluded  that 
the  space  acquisition  process  was  characterized  by  “excessive  technical 
and  schedule  risk,”  as  well  as  “unrealistic  budgets  and  unexecutable 
programs”  (Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  2003,  p.  2).  Space  acquisition  reform  was 
seen  by  many  as  crucial  to  gain  congressional  support  for  increased 
spending  on  space  programs.  As  one  observer  noted,  “[y]ears  of 
chronic  cost  overruns,  delays  and  technical  problems  have  made  law¬ 
makers  leery  of  moving  too  fast  on  ambitious  Air  Force  space  con¬ 
cepts”  (Pasztor  and  Karp,  2005). 

A  unique  set  of  formal  acquisition  regulations,  guidance,  and  in¬ 
structions,  separate  from  the  standard  DoD  5000  series,  governs  the 
acquisition  of  military  space  systems.  The  Young  Panel  findings  led 
to  a  thorough  revision  of  these  policies  under  direct  Air  Force  super¬ 
vision.  In  October  2003,  less  than  six  months  after  the  publication  of 
the  Young  Panel  findings,  DoD  issued  its  revamped  National  Secu¬ 
rity  Space  Acquisition  Policy  Document  Number  03-01  (NSSAP 
03-01)  through  the  Office  of  the  Under  Secretary  of  the  Air  Force  for 
Space  Acquisition.23  Because  of  the  unique  nature  of  space  systems, 
NSSAP  03-01  differs  in  many  significant  respects  from  the  acquisi- 


23  U.S.  Department  of  Defense  (2003b).  A  revised  and  updated  version  was  issued  about  a 
year  later,  which  changed  acquisition  Phase  C  (build,  test,  and  launch)  to  a  design  comple¬ 
tion  phase,  and  added  a  new  build  and  operations  Phase  D.  However,  the  sections  directly 
relevant  to  EA  were  not  altered  (U.S.  Department  of  Defense,  2004). 
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tion  guidance  laid  out  in  the  new  DoD  5000  series  issued  in  May. 
However,  in  one  key  respect  of  special  interest  here,  both  sets  of 
guidance  documents  are  identical:  They  both  emphasize  that  evolu¬ 
tionary  acquisition  is  the  preferred  strategy  for  DoD  acquisition. 
NSSAP  03-01  includes  the  same  definitions  with  the  same  distinc¬ 
tions  for  spiral  development  and  incremental  development  as  DoDI 
5000.2. 

NSSAP  03-01  explicitly  requires  a  considerable  amount  of  spe¬ 
cific  documentation  on  each  program’s  use  of  EA.  It  mandates  that 
each  program  manager  describe  the  EA  strategy  that  will  be  used  in 
the  space  system  acquisition  strategy  (SSAS),  a  crucial  document  that 
must  be  updated  for  each  major  phase  of  the  program.  For  major 
DoD  space  systems,  the  reasons  that  an  EA  strategy  is  adopted  or  not 
adopted  must  also  be  carefully  justified  and  documented.24  The  re¬ 
quired  program  cost  estimates  must  include  “all  blocks  that  will  make 
up  an  evolutionary  acquisition  to  the  extent  that  subsequent  blocks 
can  be  defined”  (DoD,  2004,  p.  2). 25  In  addition,  the  rationale  for 
dividing  the  program  into  specific  spirals  or  increments  must  be  ex¬ 
plained.  At  each  key  decision  point  (KDP)  in  a  program,  the  program 
office  is  required  to  prepare  an  integrated  program  summary  (IPS), 
which  constitutes  the  spiral  development  plan  for  programs  using  the 
spiral  development  process.26 


24  According  to  NSSAP  03-01,  a  major  space  defense  acquisition  program  is  defined  as  one 
in  which  the  eventual  expenditure  on  research,  development,  test,  and  evaluation  (RDT&E) 
is  expected  to  exceed  $356  million  in  fiscal  year  (FY)  2000  constant  dollars,  or  $2,190  billion 
in  FY  2000  constant  dollars  on  procurement,  or  that  is  designated  as  “major”  by  the  desig¬ 
nated  DoD  authority.  The  dollar  requirements  were  established  by  statute  in  FY  1990  dol¬ 
lars.  The  definition  is  established  by  10  USC  2430  and  is  the  same  as  the  DoD  definition  for 
an  Acquisition  Category  I  (ACAT  I)  program.  The  dollar  amounts  have  been  updated  in 
accordance  with  procedures  outlined  by  the  law  (DoD,  2004,  Section  3.1.1). 

2^  While  this  provision  appears  to  provide  flexibility  for  the  need  to  cost  spirals  precisely  far 
in  the  future,  expensive  high-profile  space  programs  such  as  SBR  tend  to  be  held  to  a  much 
higher  and  stricter  standard  by  Congress.  See  Chapter  Two. 

26  KDPs  are  the  most  important  program  review  milestones  for  DoD  acquisition  programs. 
They  usually  take  place  between  major  phases  of  the  program  (such  as  between  the  study 
phase  [Phase  A]  and  the  design  phase  [Phase  B]).  At  KDPs,  overall  program  progress  is 
evaluated  by  the  designated  senior  DoD  officials,  who  make  a  decision  regarding  the  readi¬ 
ness  of  the  program  to  proceed  to  the  next  acquisition  phase. 
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Some  of  the  potential  tensions  and  challenges  posed  by  EA  for 
cost  analysts  and  acquisition  managers  are  evident  in  the  detailed 
documentation  requirements  mandated  by  NSSAP  03-01.  Most  im¬ 
portantly,  the  new  space  acquisition  policy  guidance  requires  as  much 
detailed  information  as  possible  regarding  objectives,  content,  and 
estimated  costs  of  specific  increments  or  spirals.  Yet,  in  theory,  the 
use  of  EA  makes  it  difficult  to  project  the  future  content  and  makeup 
of  specific  spirals  or  increments  beyond  the  earliest  phases  of  the  pro¬ 
gram.  This  is  particularly  true  of  a  program  that  adopts  the  spiral  de¬ 
velopment  process,  in  which  by  definition  the  end-state  requirements 
are  not  known  at  program  inception.  The  authors  therefore  flagged 
this  issue  to  investigate  in  more  detail  through  this  examination  of 
case  studies. 

Do  EA  Findings  Based  on  Space  Program  Case  Studies  Apply  to 
Other  Types  of  Programs? 

As  noted  previously,  military  space  programs  differ  in  several  impor¬ 
tant  respects  from  other  types  of  DoD  defense  acquisition  programs. 
The  differences  are  large  enough  to  justify  the  existence  of  the  sepa¬ 
rate  set  of  acquisition  regulations  referred  to  previously,  the  NSSAP 
03-01.  To  what  extent  do  the  differences  between  space  programs 
and  other  types  of  DoD  acquisition  programs  reduce  the  applicability 
of  the  findings  of  this  study  to  other  types  of  weapon  system  acquisi¬ 
tion  programs? 

This  is  a  perfectly  legitimate  and  reasonable  question,  which 
cannot  be  answered  with  100  percent  certainty.  The  authors  adopted 
four  approaches  to  dealing  with  this  question.  The  first  involved  a 
wide  ranging  series  of  interviews  with  senior  DoD  and  Air  Force  ac¬ 
quisition  officials  knowledgeable  in  EA,  in  order  to  gain  their  insights 
into  issues  and  challenges  related  to  implementing  EA  on  all  types  of 
DoD  programs  across  the  entire  spectrum  of  program  and  system 
types.  The  second  involved  the  careful  delineation  of  those  aspects  of 
space  programs  and  space  acquisition  policy  which  differ  from  other 
types  of  DoD  acquisition  programs,  and  then  offer  some  informed 
DoD  observations  on  how  these  might  affect  the  generalizability  of 
EA  findings  based  on  space  programs,  if  at  all.  The  third  entailed  a 
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comprehensive  survey  of  other  studies  and  the  open  literature  re¬ 
garding  the  application  of  EA  to  nonspace  programs,  and  a  compari¬ 
son  of  the  findings  of  these  studies  to  this  project’s  findings. 

The  fourth  and  perhaps  most  persuasive  approach  involved  re¬ 
viewing  extensive  past  published  and  current  RAND  field  research  on 
unmanned  aerial  vehicles  (UAVs),  in  particular  the  General  Atomics 
Predator  tactical  UAV  and  the  Northrop  Grumman  Global  Hawk 
High  Altitude  Long  Endurance  (HALE)  UAV,  to  see  if  the  experi¬ 
ence  with  EA  on  these  programs  is  consistent  with  what  is  found  on 
space  programs.  The  reason  these  UAV  cases  are  so  useful  for  com¬ 
parison  is  that  they  are  widely  considered  by  many  DoD  officials  and 
other  observers  as  key  test  cases  for  EA.  Fortunately,  RAND  has  con¬ 
ducted  extensive  acquisition  policy  research  in  the  past,  and  is  con¬ 
tinuing  such  research  in  the  present  on  these  programs.27  According 
to  an  earlier  RAND  study,  the  Predator  program  became  the  de  facto 
prototype  of  the  ACTD  process  (Thirtle,  Johnson,  and  Birkler,  1997) 
which,  as  noted  previously,  directly  led  to  the  formulation  of  the  ini¬ 
tial  DoD  EA  strategies.  A  separate  study,  conducted  by  a  senior  ac¬ 
quisition  manager  who  has  worked  on  the  Global  Hawk  program  vir¬ 
tually  since  its  inception,  recently  observed  that  “it  is  probably  not 
too  presumptuous  to  state  that  Global  Hawk  is  viewed  today  by  DoD 
leadership  as  the  test  case  for  the  determination  of  the  efficacy  of 
Evolutionary  Acquisition  as  it  relates  to  Spiral  Development”  (Pingel, 
2003,  p.  8). 

The  concluding  chapter  of  this  monograph  presents  the  authors’ 
findings  on  EA  derived  from  the  space  program  case  studies  and  the 
more  general  interviews  with  senior  DoD  experts  on  EA,  and  assesses 
their  applicability  to  other  types  of  programs  by  reviewing  and  com¬ 
paring  the  findings  of  other  studies  and  RAND  studies  on  Global 
Hawk,  Predator,  and  other  UAV  programs.  This  chapter  is  limited  to 


27  The  lead  author  of  this  monograph  is  a  coproject  leader  of  an  ongoing  RAND  study 
sponsored  by  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition  on  UAVs,  with  a  special 
focus  on  Global  Hawk  and  Predator,  and  is  responsible  for  assessing  the  effects  of  EA  on 
these  programs  for  acquisition  managers  and  cost  analysts.  Past  major  published  RAND 
assessments  of  these  and  other  UAV  programs  include  Drezner  and  Leonard  (2002)  and 
Thirtle,  Johnson,  and  Birkler  (1997). 
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a  quick  review  of  how  space  programs  differ  from  other  types  of  DoD 
acquisition  programs  and  how  NSSAP  03-01  differs  from  the  DoD 
5000  regulations  with  respect  to  EA,  and  offers  a  few  observations  on 
the  relevance  of  these  differences. 

Based  on  this  case  study  research,  interviews  with  senior  DoD 
officials,  and  review  of  the  open  literature,  the  authors  conclude  that 
space  programs  differ  from  other  acquisition  programs  in  at  least  four 
important  respects: 

1.  Space  programs  are  characterized  by  very  small  procurement 
numbers  of  space  vehicles  (SVs).  According  to  DoD,  “  [s]  atellite 
programs  produce  from  1-25  satellites — 6  being  average,”  com¬ 
pared  to  procurement  numbers  in  the  hundreds  or  even  thousands 
for  such  items  as  tactical  fighter  aircraft  or  smart  munitions  (U.S. 
General  Accounting  Office,  2003,  Appendix  III). 

2.  All  SV  component  testing  cannot  be  done  in  an  operational  envi¬ 
ronment  (space),  because  of  the  high  cost  of  space  launches  and 
the  limited  number  of  operational  SVs  in  any  system. 

3.  A  larger  percentage  of  total  program  expenditures  take  place  in  the 
early  phases  of  a  space  acquisition  program  compared  to  other  ac¬ 
quisition  programs  (Adams,  2004). 

4.  Space  program  technology  development  continues  on  longer  in 
the  procurement  process  than  is  typical  (or  desirable)  in  other 
types  of  procurement  programs,  according  to  the  U.S.  Govern¬ 
ment  Accountability  Office  (GAO),  formerly  the  General  Ac¬ 
counting  Office.  This  situation,  according  to  GAO,  appears  to 
have  been  formalized  in  the  NSSAP  03-01  regulations.  However, 
DoD  strongly  disputes  GAO’s  interpretation  of  NSSAP  03-01 
(U.S.  General  Accounting  Office,  2003). 

While  these  differences  appear  to  be  real  (at  least  the  first  three), 
it  is  unclear  what  impact,  if  any,  they  have  on  the  applicability  of  EA 
lessons  derived  from  space  program  case  studies  to  other  types  of 
DoD  programs.  While  the  2003  GAO  report  strongly  criticizes 
NSSAP  03-01  for  promoting  a  high-risk  approach  to  acquisition  by 
allowing  the  beginning  of  the  production  stage  to  overlap  with  on- 
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going  technology  development,  DoD  vigorously  disputes  GAO’s 
findings  and  insists  that  NSSAP  03-01  enshrines  and  promotes  the 
same  basic  EA  principles  that  underpin  DoD  5000.2. 

While  DoD’s  response  to  the  2003  GAO  report  recognizes  dif¬ 
ferences  between  space  and  other  programs,  it  stresses  that  “NSS  Ac¬ 
quisition  Policy  03-01  and  DoDI  5000.2  are  consistent  in  their  in¬ 
tent,”  particularly  in  the  area  of  evolutionary  acquisition  and  the 
importance  of  proof  testing  technologies,  reducing  technological  risk, 
incorporating  feedback  from  the  user,  and  implementing  regular  pro¬ 
gram  and  cost  reviews,  all  key  components  of  an  EA  approach  as  laid 
out  in  DoDI  5000.2. 

It  can  be  argued  that  the  only  significant  difference  between  the 
structuring  of  space  programs  based  on  NSSAP  03-01  and  other  pro¬ 
grams  based  on  DoDI  5000.2  is  that  two  key  decision  points  (Mile¬ 
stones  B  and  C)  come  earlier  for  space  programs,  as  shown  in  Figure 
1.6.  Furthermore,  there  are  more  major  program  reviews  in  the  early 
ph  ases  of  a  space  program,  ensuring  user  feedback  and  technology 
maturity.28 

NSSAP  03-01  emphasizes  that  space  programs  by  their  very  na¬ 
ture  must  be  conducted  with  an  EA  approach,  and  have  for  some 
time  been  carried  out  that  way.  The  document  includes  the  same 
wording  as  DoDI  5000.2  on  EA,  and  adds  that  “[e] volutionary  ac¬ 
quisition  has  been  a  cornerstone  for  space  system  development  since 
the  early  1960’s”  (DoD,  2004,  p.  9).  All  space  programs  using  EA  are 
required  to  include  a  formal  description  of  their  EA  strategy  in  their 
Acquisition  Strategy  documentation  and  a  spiral  development  plan  in 
their  integrated  program  summary  (IPS). 


28  These  include  the  system  requirement  review  (SRR)  in  the  later  part  of  Phase  A,  the  sys¬ 
tem  design  review  (SDR)  at  the  beginning  of  Phase  B,  the  preliminary  design  review  (PDR) 
in  the  middle  of  Phase  B,  and  the  critical  design  review  (CDR)  at  the  transition  from  Phase 
B  to  Phase  C. 
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Figure  1.6 

DoDI  5000.2  Versus  NSSAP  03-01:  Acquisition  Phase  Comparison 


SOURCE:  U.S.  Department  of  Defense  (2004,  Figure  1,  p.  5). 

NOTES:  SRR  =  system  requirements  review;  SDR  =  system  design  review;  PDR  =  preliminary  design  review;  CDR  = 
critical  design  review.  The  December  2004  revision  of  NSSAP  03-01  significantly  changed  the  space  acquisition 
phasing  laid  out  in  the  earlier  document,  apparently  in  an  effort  to  reduce  space  program  risk.  In  the  updated 
NSSAP  03-01,  Phase  C  was  split  into  two  separate  new  phases.  Phase  C  became  the  complete  design  phase  and  a 
new  phase  was  added  that  was  designated  Phase  D  and  called  build  and  operations.  Several  key  design  review 
events  are  moved  in  the  new  phasing,  apparently  in  response  to  criticism  that  the  October  2003  version  permitted 
programs  to  enter  into  the  acquisition  phase  too  early  in  the  development  process,  thus  raising  program  risk.  In 
the  new  version,  the  SDR  is  moved  back  to  the  end  of  Phase  A,  and  thus  becomes  part  of  the  entry  requirements 
for  Milestone  B.  The  PDR  is  moved  to  the  end  of  Phase  B  and  becomes,  in  effect,  part  of  the  entry  requirements 
for  Milestone  C.  The  CDR  is  moved  far  to  the  right  and  immediately  precedes  the  new  build  approval  milestone 
and  entry  into  the  new  Phase  D.  This,  in  effect,  provides  more  time  to  mature  the  design  and  technologies  before 
committing  to  a  final  design. 


RAND  MG431-1.6 


38  Evolutionary  Acquisition 


At  the  end  of  the  day,  however,  the  authors  are  inclined  to  agree 
with  the  DoD  observation  included  in  its  response  to  GAO  that  “it  is 
difficult  to  accurately  compare  NSS  Acquisition  Policy  03-01  to  the 
new  DoDI  5000.2,  since  it  is  too  early  to  judge  what  effect  the  new 
5000  series  will  have  on  traditional  acquisition  challenges”  (DoD, 
2003b,  p.  23). 

Based  on  the  quick  review  here  of  how  space  programs  differ 
from  other  types  of  DoD  acquisition  programs,  and  how  the  formal 
NSSAP  03-01  policy  differs  from  the  DoD  5000  regulations  with 
respect  to  EA,  the  authors  conclude  that  based  on  these  factors  there 
is  no  consensus  and  little  hard  evidence  either  way  regarding  the  ap¬ 
plicability  of  space  program  lessons  learned  on  EA  to  other  types  of 
DoD  defense  acquisition  programs. 

It  is  for  this  reason  that  the  authors  rely  heavily  in  the  last  chap¬ 
ter  of  this  monograph  on  a  comparison  of  these  findings  from  space 
programs  to  past  and  ongoing  acquisition  research  on  EA  as  applied 
to  Global  Elawk  and  Predator  as  well  as  other  UAVs,  and  to  other 
available  research  that  has  assessed  real-world  experience  with  EA  on 
nonspace  programs.  It  is  from  this  evidence  that  the  authors  conclude 
that  the  findings  from  space  programs  are  reasonably  robust  and 
largely  applicable  to  a  much  broader  spectrum  of  DoD  programs. 
The  underpinnings  for  this  conclusion  are  discussed  in  detail  in 
Chapter  Three,  especially  in  “Applicability  of  EA  Findings  from 
Space  Programs  to  Other  DoD  Defense  Acquisition  Programs.” 

Case  Study  Selection  and  Methodology 

As  noted  previously,  the  authors  selected  the  following  five  space  pro¬ 
grams  as  case  studies  for  this  research  effort. 

•  Space-Based  Space  Surveillance  (SBSS)  System 

•  Rapid  Attack  Identification,  Detection,  and  Reporting  System 
(RAIDRS) 

•  Global  Positioning  Satellite  (GPS)  III 

•  Space-Based  Radar  (SBR) 

•  Kinetic  Energy  Interceptor  (KEI) 
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The  goal  was  to  review  systematically  the  five  selected  programs 
to  gain  insights  and  lessons  learned  into  how  cost  analysts  and  acqui¬ 
sition  managers  are  actually  implementing  EA  in  the  real  world.  The 
authors  selected  these  programs  based  on  the  recommendations  of 
cost  analysts  and  acquisition  managers  at  the  Air  Force  Space  and 
Missile  Systems  Center  (SMC)  and  elsewhere,  whom  the  authors  in¬ 
formally  surveyed.29  A  consensus  of  officials  consulted  agreed  that 
this  collection  of  cases  was  a  good  representative  sample  of  a  variety  of 
different  types  of  space  programs  attempting  to  implement  EA  as 
mandated  in  the  new  DoD  space  acquisition  policy  guidance.  These 
cases  are  representative  of  a  wide  spectrum  of  Air  Force  space  pro¬ 
grams  in  terms  of  scale  of  effort  (from  relatively  modest  programs 
such  as  RAIDRS  to  potentially  large  programs  such  as  SBR),  and  in 
specific  details  of  EA  implementation  strategies,  as  well  as  other  fac¬ 
tors.  In  addition,  these  cases  were  among  those  whose  program  offices 
agreed  to  participate  in  the  case  study  effort. 

The  U.S.  Air  Force,  through  its  Space  and  Missile  Systems  Cen¬ 
ter  (SMC),  is  the  lead  acquisition  management  authority  for  the  first 
four  programs.  The  KEI  program,  however,  is  in  many  respects  a  spe¬ 
cial  case.  It  is  managed  by  DoD’s  Missile  Defense  Agency  (MDA), 
not  the  U.S.  Air  Force.  MDA  acquisition  programs  are  integrated 
into  a  unique,  overall,  integrated  missile  defense  acquisition  strategy, 
but  are  implemented  under  the  guidance  of  the  standard  DoD  5000 
series  of  acquisition  guidance  documents.  The  authors  selected  the 
KEI  program  because  it  is  a  space  weapon  system  program  that  is 
being  implemented  in  part  through  the  rigorous  application  of  DoD- 
mandated  EA  concepts.  The  program  is  also  of  special  interest  be¬ 
cause  the  Program  Manager,  Terry  Little,  is  a  well-known  pioneering 
acquisition  reformer  who  has  led  some  of  the  most  innovative  recent 


29  Based  on  expert  recommendations,  the  authors  had  originally  also  intended  to  include  a 
sixth  program,  the  transformational  satellite  (TSAT),  which  is  part  of  the  transformational 
communication  program.  Limitations  in  the  research  schedule  and  resources,  however,  pre¬ 
vented  us  from  including  TSAT  as  a  full  case  study.  Therefore,  it  is  not  discussed  in  this 
monograph,  although  it  has  also  adopted  an  EA  approach. 


40  Evolutionary  Acquisition 


acquisition  programs  in  the  U.S.  Air  Force,  and  has  been  directly  in¬ 
volved  in  the  development  of  the  Air  Force  approach  to  EA.30 

Organization  of  This  Monograph  and  Important  Qualifications 

Chapter  Two  of  this  monograph  begins  with  a  discussion  of  over¬ 
arching  issues  raised  by  all  the  case  studies,  then  proceeds  to  a  brief 
description  of  the  specific  case  studies’  program  structures  and  acqui¬ 
sition  approaches,  and  ends  with  a  presentation  of  the  findings  from 
the  case  studies  of  relevance  to  cost  analysts  and  acquisition  managers. 

Chapter  Three  presents  a  summary  overview  of  the  authors’ 
findings  and  some  concluding  observations. 

As  discussed  briefly,  it  is  legitimate  to  question  whether  findings 
regarding  EA  based  on  space  program  case  studies  are  fully  relevant  to 
other  types  of  major  DoD  acquisition  programs.  The  authors  believe 
these  findings  are  reasonably  robust  across  the  whole  spectrum  of 
major  DoD  programs.  The  reasoning  behind  this  conclusion  is  dis¬ 
cussed  in  detail  in  Chapter  Three. 

It  is  also  important  to  note  here,  however,  that  the  space  pro¬ 
grams  examined  as  case  studies  are  all  in  the  very  earliest  stages  of  the 
acquisition  process.  Therefore,  the  lessons  learned  that  the  authors 
derived  from  them  must  be  considered  tentative  and  treated  as  provi¬ 
sional.  In  addition,  it  is  important  to  note  that  the  programs  varied  in 
size  and  complexity,  as  well  as  in  the  interpretation,  definition,  and 
application  of  the  rather  general  guidance  on  the  evolutionary  acqui¬ 
sition  strategies  and  processes  provided  by  the  DoD  5000  series  and 
NSSAP  03-01.  The  authors  provide  a  taxonomy  of  the  different  gen¬ 
eral  types  of  programs  in  the  list  of  case  studies  (see  Chapter  Two, 

pp.  45-46). 

Another  significant  qualification  the  reader  should  keep  in  mind 
is  that  this  study  focuses  almost  exclusively  on  the  acquisition  man- 


30  Terry  Little  was  the  influential  program  manager  in  the  1990s  of  two  key  Air  Force  acqui¬ 
sition  reform  pilot  programs,  the  joint  direct  attack  munition  (JDAM)  and  the  joint  air-to- 
surface  strike  missile  (JASSM) .  Later,  he  became  director  of  the  Acquisition  Center  of  Excel¬ 
lence  (ACE)  under  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition.  For  accounts  of 
these  and  other  Air  Force  acquisition  reform  programs  and  their  implications  for  cost  ana¬ 
lysts,  see  Lorell  et  al.  (2000)  and  Lorell  and  Graser  (2001). 
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agement  and  cost  analysis  community.  The  authors  did  not  consult 
with  or  interview  representatives  of  the  user  communities.  Therefore, 
the  findings  are  heavily  slanted  toward  the  perspective  and  problems 
of  the  acquisition  community.  The  user  communities  may  have  a 
completely  different  take  on  these  programs.  Understanding  their 
perspective  is  of  course  crucial  to  an  overall  high-level  assessment  of 
the  efficacy  of  EA  to  the  Air  Force  as  a  fighting  organization.  Since 
the  charter  for  this  study  directed  the  authors  to  assess  the  implica¬ 
tions  of  EA  for  the  acquisition  and  cost  community,  the  authors  leave 
such  an  overall  higher-level  assessment  of  EA  to  other  analysts  in  the 
future.  Hopefully  this  study  provides  at  least  some  useful  input  for 
such  a  future  study. 


CHAPTER  TWO 

Evolutionary  Acquisition  in  Practice: 
Five  Case  Studies 


Introduction 

This  chapter  presents  brief  summaries  of  the  case  studies  of  the  five 
space  system  acquisition  programs  selected  as  examples  of  the  new 
approach  to  evolutionary  acquisition.  As  noted  in  Chapter  One,  these 
programs  are  as  follows: 

•  SBSS  system 

•  RAIDRS 

•  GPS  III 

•  SBR 

•  KEI.1 

Before  reviewing  the  individual  case  studies,  the  monograph 
briefly  discusses  various  general  characteristics  of  all  the  programs,  as 
well  as  some  crosscutting  issues  particularly  relevant  to  acquisition 
management  and  cost  analysis.  First,  the  monograph  looks  at  cross¬ 
cutting  acquisition  management  issues.  Then  it  examines  crosscutting 
cost  analysis  issues.  The  five  principal  case  studies  are  then  summa¬ 
rized  and  briefly  reviewed.  The  chapter  ends  with  some  additional 
observations  and  findings  drawn  from  the  specific  case  studies. 


1  As  mentioned  previously,  the  authors  initially  planned  to  include  a  sixth  program,  the 
transformational  satellite  (TSAT)  within  the  transformational  communication  program. 
Because  of  resource  and  schedule  constraints,  the  authors  removed  this  program  from  the  list 
of  case  studies. 
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Crosscutting  Acquisition  Management  Issues 

Program  Maturity 

It  is  important  to  note  that  all  five  of  the  case  study  programs  studied 
in  detail  were  in  very  early  program  stages  at  the  time  the  authors  in¬ 
terviewed  program  officials  and  collected  information.2  This  was  un¬ 
avoidable  given  that  the  authors  wanted  to  examine  programs  that 
were  implementing  the  new  DoD  EA  guidelines  from  scratch,  since 
those  guidelines  were  not  issued  in  final  form  until  well  into  2003 
(and  then  revised  at  the  end  of  2004).  All  five  programs  were  thus  in 
the  presystems  acquisition  phase,  focused  primarily  on  pre-Key  Deci¬ 
sion  Point  A  (KDP-A)  or  study  Phase  A  activities.  Thus  all  the  pro¬ 
grams  were  involved  in  basic  concept  refinement  issues  and  in  archi¬ 
tecture  development  and  technology  refinement.  Given  the  very  early 
ph  ase  of  most  of  these  programs,  it  is  not  surprising  that  many  basic 
uncertainties  remain  regarding  program  structure,  requirements,  ac¬ 
quisition  approach,  and  other  fundamental  issues.  Therefore,  the 
broad  applicability  of  the  lessons  learned  and  challenges  encountered 
to  date  on  these  programs  for  acquisition  managers  must  be  viewed 
with  extreme  caution.  Nonetheless,  the  authors  believe  that  enough 
experience  had  been  gained  on  these  programs  dealing  with  many  of 
the  basic  challenges  posed  by  EA  to  provide  useful  insights  and  les¬ 
sons  for  future  programs. 

Variations  in  Evolutionary  Acquisition  Program  Approach 

The  authors  found  that  the  case  studies  varied  widely  in  terms  of  the 
aspects  of  EA  that  they  emphasize,  and  their  application  of  EA  termi¬ 
nology  and  principles.  Sometimes  variations  existed  even  within  pro¬ 
grams.  For  example,  at  least  one  program  originally  labeled  each  of  its 
EA  developmental  phases  as  “spirals,”  presumably  reflecting  the 
DoD-preferred  EA  process  of  spiral  development.  Later,  these  phases 
were  called  “increments”  rather  than  “spirals.” 


2  Almost  all  interviews  and  information  collection  on  case  studies  took  place  during  the  first 
nine  months  of  2004. 
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This  research  indicates  that  variations  in  the  use  of  terminology 
have  essentially  two  causes.  The  first  is  the  continuing  ambiguity  and 
confusion  over  the  basic  terminologies  used  in  EA,  even  following  the 
attempted  clarifications  promulgated  in  the  new  DoD  5000  guidance 
issued  in  2003.  Perhaps  more  significant,  this  research  suggests  that 
all  the  programs  have  over  time  begun  to  move  away  from  the  pre¬ 
ferred  “spiral”  approach  toward  an  “incremental”  approach,  as  de¬ 
fined  in  the  5000  guidance  documents.  This  appears  to  be  a  response 
to  a  variety  of  pressures  emanating  from  the  existing  political,  statu¬ 
tory,  and  regulatory  acquisition  environments.  Most  important,  issues 
soon  emerged  on  most  of  these  programs  regarding  the  definition  of 
requirements  and  cost  estimating  approaches  for  specific  EA  devel¬ 
opment  phases,  which  tended  to  encourage  movement  away  from  a 
spiral  development  approach  and  toward  an  incremental  development 
approach.  More  on  these  issues  is  said  below  and  elsewhere. 

Program  Scale,  Complexity,  and  Other  Factors:  A  Taxonomy 

The  authors  also  noted  that  such  basic  program  characteristics  as  pro¬ 
gram  size  (in  terms  of  budget  and  cost),  technological  and  program¬ 
matic  complexity,  nature  of  the  user  communities,  and  other  program 
characteristics,  varied  considerably  among  case  studies.  Perhaps  the 
only  feature  common  to  all  of  them  is  that  none  has  yet  moved  past 
the  concept  development  phase,  as  mentioned  previously.  The 
authors  developed  a  taxonomy  that  divides  the  case  study  into  three 
categories  for  analytical  purposes.  The  three  categories  are  loosely 
based  on  relative  program  cost,  complexity,  and  acquisition  style. 

The  first  two  programs  considered,  SBSS  and  RAIDRS,  are  rela¬ 
tively  modest,  low-cost,  low-complexity  acquisition  programs  that  so 
far  have  been  conceptually  structured  into  only  two  basic  increments. 
At  this  stage,  the  first  increment  is  by  far  the  most  important.  For  the 
initial  increment  for  each  program,  the  end  requirements  are  rela¬ 
tively  well  defined,  allowing  them  to  be  treated  effectively  as  stand¬ 
alone  programs.  Concepts  for  the  final  end  system  after  a  second  in¬ 
crement  are  both  more  ambitious  and  much  less  well  defined  because 
objective  end  requirements  are  still  quite  uncertain.  In  each  case,  it  is 
possible  that  the  initial  system  from  the  first  planned  increment  will 
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evolve  in  a  direction  not  now  anticipated,  leading  to  the  possible 
launch  at  some  time  in  the  future  of  an  entirely  new  program  to  meet 
the  objective  requirements  now  envisioned  for  the  second  increment 
(once  those  requirements  are  clearly  defined). 

The  next  two  case  study  programs,  GPS  III  and  SBR,  are  more 
complex  and  potentially  much  higher  cost  than  SBSS  and  RAIDRS. 
Notionally  they  are  structured  into  multiple  increments.3  Just  how 
many  increments  remained  uncertain  in  mid-2004,  because  the 
threshold  and  objective  requirements,  as  well  as  the  specific  content 
and  structuring  of  many  of  the  increments,  was  still  in  the  process  of 
being  determined.  Desired  objective  capabilities  for  the  two  end  sys¬ 
tems,  however,  have  largely  been  determined  by  their  many  potential 
user  groups. 

The  final  case  study  program,  KEI,  is  in  many  respects  a  unique 
case  because  of  the  multitude  of  innovative  acquisition  measures  be¬ 
ing  applied  to  the  program  and  the  nonstandard  aspects  of  all  MDA 
major  acquisition  programs.  Not  only  does  the  KEI  program  apply 
EA,  but  it  employs  numerous  other  acquisition  reform  strategies  such 
as  price-based  and  capabilities-based  acquisition.4 

The  authors  found  some  subtle  differences  in  the  findings  and 
lessons  learned  based  on  the  category  of  program  as  outlined  in  the 
taxonomy  discussed  here.  These  differences,  however,  are  minor,  and 
will  be  noted  later  when  relevant. 

Managing  the  Capabilities  and  Requirements  Processes 

All  of  these  case  studies  point  to  the  conclusion  that  the  capabilities 
and  requirements  definition  and  management  processes  are  major 
challenges  in  all  EA  programs.  All  major  acquisition  programs  must 
confront  uncertainties  and  disagreement  about  necessary  capabilities 
and  system  requirements  to  provide  the  capabilities.  This  situation  is 
made  much  more  challenging  and  complex  by  EA.  This  is  because 


3  In  2005,  SBR  was  restructured  essentially  as  a  technology  demonstration  program.  See 
specific  case  study  below. 

4  For  an  assessment  of  price-based  acquisition  (PBA),  see  Lorell,  Graser,  and  Cook  (2005). 
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EA  programs  tend  to  treat  each  separate  increment  or  spiral  much 
like  a  separate  program,  which  requires  its  own  separate  definition  of 
capabilities  and  requirements.  This  tendency  is  reinforced  by  the  new 
DoD  5000  document  guidance  and  congressional  guidance  that  re¬ 
quire  each  increment  to  pass  through  most  of  the  same  formal  mile¬ 
stones,  program  reviews,  and  documentation  as  the  overall  program. 
At  the  same  time,  each  increment  is  not  truly  a  stand-alone  program, 
but  rather  a  part  of  a  much  larger  single  effort.  As  a  result,  acquisition 
managers  have  to  justify  the  operational  usefulness  of  the  end  product 
of  each  increment  as  part  of  an  operationally  useful  threshold  capa¬ 
bility,  while  at  the  same  time  justify  the  need  to  continue  with  fol¬ 
low-on  increments  in  order  to  meet  a  needed  objective  capability  fur¬ 
ther  down  the  road.  Such  an  approach  also  requires  the  development 
of  a  carefully  laid  out  plan  that  maps  the  specific  path  through  several 
increments  that  achieve  first  the  threshold  capability  and  end  up  with 
the  desired  objective  capability. 

Thus  the  application  of  EA  in  the  current  regulatory  environ¬ 
ment  requires  program  managers  early  in  a  program  to  define  the 
specific  content  and  structure  of  each  increment  or  spiral,  the  content 
and  timing  of  the  threshold  and  objective  capabilities,  and  the  precise 
path  to  the  end  goal.  In  the  cases  examined,  the  authors  found  that 
system  program  offices  (SPOs)  tended  to  rely  heavily  on  competitive 
contractor  studies  to  assist  them  in  these  tasks.  Credible  assessment 
and  future  projection  of  the  relative  maturity  of  key  enabling  tech¬ 
nologies  are  among  the  most  demanding  and  difficult  aspects  of  these 
studies.  As  many  program  officials  interviewed  confirmed,  DoD’s 
transition  from  requirements-based  to  capabilities-based  acquisition 
approaches  is  greatly  increasing  technical  and  management  flexibility, 
but  it  is  also  increasing  program  complexity  and  expanding  the  up¬ 
front  work  load  on  the  SPO  and  contractors. 

Another  related  area  of  concern  encountered  from  program  offi¬ 
cials  during  the  interviews  regarded  the  challenge  of  controlling  re¬ 
quirements  and  capabilities  creep  in  EA  programs.  The  2003  Young 
Panel  Report  identified  “undisciplined  definition  and  uncontrolled 
growth  in  system  requirements”  as  one  of  the  “basic  reasons  for  the 
significant  cost  growth  and  schedule  delays  in  national  security  space 
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programs”  (DoD,  2003b,  p.  2).  While  all  major  defense  acquisition 
programs  must  cope  with  this  issue,  space  programs  confront  a  par¬ 
ticularly  challenging  environment  because  of  the  multiple  and  con¬ 
tinuously  proliferating  number  of  user  communities.  It  is  not  sur¬ 
prising,  then,  that  many  program  officials  interviewed  expressed 
concerns  about  the  open-ended  and  more  flexible  nature  of  the  re¬ 
quirements  process,  which  is  a  key  attribute  of  the  spiral  development 
process,  with  constant  feedback  loops  from  user  communities.  In 
some  cases,  programs  chose  to  establish  institutionalized  and  rigorous 
requirements  control  mechanisms,  and  sought  to  limit  severely  user 
feedback  loops  into  the  requirements  process  until  after  developmen¬ 
tal  test  and  evaluation  (DT&E). 

Crosscutting  Cost  Analysis  Issues 

EA  Complicates  "Truth  in  Costing" 

The  Young  Panel  report  placed  a  strong  emphasis  on  the  need  to  im¬ 
prove  the  quality  and  realism  of  early  program  cost  estimates  in  na¬ 
tional  security  space  programs.  The  Panel  concluded  that  the  “space 
acquisition  system  is  strongly  biased  to  produce  unrealistically  low 
cost  estimates  throughout  the  process.”  This  problem  in  turn  “leads 
to  unrealistic  budgets  and  unexecutable  programs”  (DoD,  2003b,  p. 
2).  In  response  to  the  Young  Panel  findings,  NSSAP  03-01  places 
considerable  emphasis  on  enhanced  mechanisms  for  improved  cost 
estimating  in  space  programs,  including  formation  of  independent 
cost  analysis  teams  (ICATs),  and  the  formal  requirement  for  inde¬ 
pendent  cost  assessments  (ICAs)  or  independent  cost  estimates  (ICEs) 
at  various  program  key  decision  points  (KDPs).5  The  objective  of 
these  measures,  in  the  words  of  one  program  official  interviewed,  is  to 
achieve  “truth  in  costing.” 

The  Young  Panel  Report  findings  and  NSSAP  03-01  have  en¬ 
couraged  a  high  level  of  sensitivity  to  cost  estimating  issues  among 


5  For  details,  see  U.S.  Department  of  Defense  (2004,  Appendix  Three). 
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program  managers  involved  in  the  case  studies  examined.  In  particu¬ 
lar,  several  of  the  programs  examined  emphasized  that  a  much 
stronger  effort  was  being  made  by  program  offices  to  include  all 
ground,  user,  and  support  elements,  as  well  as  complete  life-cycle  cost 
(LCC)  factors,  in  the  overall  cost  estimating  process,  in  order  to 
achieve  the  goal  of  full  truth  in  costing. 

Unfortunately,  many  program  officials  interviewed  noted  that 
EA  makes  the  renewed  attempts  to  achieve  truth  in  costing  much 
more  complex.  The  Young  Panel  noted  that  in  addition  to  the  prob¬ 
lems  of  requirements  creep  and  requirements  changes  touched  on 
above,  several  other  factors  are  basic  reasons  for  cost  growth  on  space 
programs.  These  factors  include  contractor  buy-in,  technological 
overoptimism  among  both  contractor  and  program  officials,  flawed 
technological  maturity  assessments,  and  inadequate  assessment  of 
overall  program  risk.  Accurately  assessing  these  issues  can  become 
much  more  complex  on  an  EA  program,  in  the  opinion  of  many  offi¬ 
cials  interviewed.  This  is,  of  course,  because,  by  definition,  the  end 
requirements  in  spiral  development  are  not  known  at  the  time  of  pro¬ 
gram  inception.  In  addition,  as  noted  above,  program  officials  and 
contractors  discovered  early  on  that  defining  the  precise  content  and 
structure  of  specific  spirals  or  increments,  an  acceptable  level  of 
threshold  requirements  and  capabilities  for  initial  increments,  and  the 
overall  path  to  the  final  objective  requirements  and  capabilities,  were 
proving  to  be  challenging  tasks  indeed.  Some  program  officials  ob¬ 
served  that  the  uncertainties  and  complexities  involved  in  this  process 
greatly  increased  the  difficulties  confronted  by  the  cost  analysts,  and 
complicated  the  attempts  to  maximize  truth  in  costing  through  the 
development  of  higher  quality  cost  estimates. 

EA  Necessitates  Evolutionary  or  Incremental  Cost  Estimating 

The  challenges  discussed  previously  have  led  program  managers  and 
cost  analysts  to  conclude  that  EA,  especially  if  implemented  through 
the  spiral  development  process,  requires  the  use  of  evolutionary  or 
incremental  cost  estimating.  In  other  words,  given  the  uncertainties 
regarding  the  detailed  content  of  future  increments  or  spirals,  it  must 
be  accepted  that  cost  estimates  have  to  evolve  and  be  updated  con- 
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tinuously  as  the  program  structure  and  requirements  unfold  and  ma¬ 
ture  over  time. 

This  issue,  along  with  other  mandated  program  requirements, 
has  tended  to  encourage  program  managers  to  move  away  from  the 
spiral  development  process,  where  end  requirements — even  for  the 
initial  increment — may  not  be  known  at  program  inception,  and  to¬ 
ward  an  incremental  development  approach  where  requirements  are 
well  defined  at  program  inception,  thus  simplifying  the  task  of  the 
cost  analysts  and  permitting  greater  fidelity  to  the  truth  in  costing 
concept.  The  existing  statutory,  regulatory,  and  oversight  reporting 
requirements  have  pushed  program  managers  in  this  direction.  With¬ 
out  firm  requirements  and  a  clearly  mapped  out  program  structure 
early  in  a  program — at  the  very  least  for  the  first  major  increment — 
programs  may  appear  too  open-ended  and  ill-defined,  and  thus  pos¬ 
sibly  more  subject  to  the  factors  identified  by  the  Young  Panel  as 
leading  to  unacceptable  cost  growth.  Such  concerns  appear  to  have 
effectively  ended  the  use  of  spiral  development  as  originally  envi¬ 
sioned  in  the  case  studies  examined,  although  acquisition  managers 
insist  that  the  concept  had  already  been  modified  to  apply  only 
within  specific  and  well-defined  increments  with  clear  and  established 
requirements,  rather  than  across  increments. 

Focus  of  Cost  Estimating  Effort  Is  on  the  First  Increment 

Given  the  factors  discussed  previously,  the  application  of  EA  in  many 
of  the  programs  examined  has  led  to  the  de  facto  treatment  for  cost¬ 
ing  purposes  of  the  initial  program  increments  or  spirals  as,  in  effect, 
equivalent  to  stand-alone  programs.  This  is  because  often  only  the 
first  increment  has  achieved  the  level  of  detail  regarding  requirements 
and  other  program  details  that  permits  application  of  rigorous  bot¬ 
tom-up  cost  estimating  methodologies.  With  all  the  programs  exam¬ 
ined  in  early  concept  formulation  phases,  follow-on  increments  are 
defined  (if  at  all)  to  a  level  of  detail  that  permits  only  high-level  esti¬ 
mates,  which  are  essentially  derived  from  the  long-term  program 
budget  plan. 
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EA  Requires  More  Up-Front  Effort  and  Cost 

The  program  officials  interviewed  generally  agreed  that  EA  requires 
more  up-front  planning,  work  effort,  and  cost  than  the  old  single- 
step-to-capability  approach.  Defining  and  coordinating  the  structure 
and  content  of  various  increments,  planning  for  the  milestone  re¬ 
quirements  for  each  increment,  assessing  and  promoting  technology 
insertion  opportunities,  and  the  need  to  develop  and  update  cost  es¬ 
timates  on  a  nearly  continuous  basis  increase  workload  for  cost  ana¬ 
lysts  and  program  managers.  The  additional  costs  for  this  increased 
workload  must  be  anticipated  and  budgeted. 

Keeping  these  crosscutting  acquisition  management  and  cost 
analysis  issues  in  mind,  the  monograph  now  briefly  reviews  and 
comments  on  the  five  principle  case  studies. 


Space  Acquisition  Case  Studies:  EA  in  Practice 

SBSS:  An  EA  Strategy  for  Rapid  Fielding  of  Threshold  Capability 

Introduction.  The  SBSS  program  is  a  relatively  straightforward, 
lower  risk,  lower  cost,  two-stage  EA  program  that  grew  out  of  a  mod¬ 
est  technology  demonstration  program.  The  main  objective  for  using 
an  EA  strategy  on  this  program  was  to  promote  a  much  more  rapid 
development  and  fielding  of  a  basic  operationally  useful  threshold 
capability  than  originally  anticipated  to  fulfill  an  urgent  and  unex¬ 
pected  near-term  requirement. 

SBSS  aims  at  providing  timely  detection,  identification,  and 
tracking  of  human-built  space  objects.  Its  goal  is  to  enhance  space 
situational  awareness  and  support  national  defensive  and  offensive 
counter  space  strategies. 

The  SBSS  program  emerged  as  a  follow-on  to  the  Midcourse 
Space  Experiment  with  Space-Based  Visible  Sensor  (MSX/SBV)  pro¬ 
gram,  a  Ballistic  Missile  Defense  Organization  (BMDO)6  technology 
demonstrator  launched  in  1996  to  test  whether  objects  in  space  could 


i;'  BMDO  has  now  become  the  Missile  Defense  Agency,  or  MDA. 
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be  identified  and  tracked  from  a  space-based  platform.  In  October 
1997,  the  Air  Force  Space  Command  (AFSPC)  formally  initiated  the 
MSX/SBV  as  an  Advanced  Concept  Technology  Demonstration 
(ACTD)  program;  the  single  space  vehicle  proved  so  useful  that  in 
2000  it  was  incorporated  into  the  Air  Force’s  largely  ground-based 
space  surveillance  network  as  a  dedicated  sensor  (Space-Based  Space 
Surveillance  Program  Office,  2004).  At  the  same  time,  the  SBSS  pro¬ 
gram  was  initiated  to  extend  and  enhance  the  capabilities  that  the 
MSX/SBV  had  demonstrated. 

When  the  SBSS  program  began,  first  launch  of  an  initial  system 
was  expected  by  fiscal  year  (FY)  2010.  Soon  after,  however,  Air  Force 
officials  became  concerned  about  the  growing  probability  of  cata¬ 
strophic  failure  of  the  SBV,  which  had  already  exceeded  its  design 
life.  In  December  2002,  the  Air  Force  Requirements  Oversight 
Council  called  for  the  fielding  of  a  replacement  for  the  SBV  by  2006, 
four  years  earlier  than  originally  anticipated. 

An  EA  strategy  appeared  to  be  ideally  suited  to  the  requirement 
of  rapidly  fielding  an  urgently  needed,  operationally  useful  basic 
threshold  replacement  capability  for  the  MSX/SBV  by  2006  to  fulfill 
the  near-term  requirement,  followed  later  by  a  more  capable  system 
to  meet  a  more  demanding  but  less  urgent  objective  requirement.  To 
reflect  this  new  EA  strategy,  the  SBSS  program  was  restructured  into 
two  increments: 

•  The  SBSS  Pathfinder,  an  SBV  replacement  intended  to  provide 
an  interim  surveillance  capability;  and 

•  The  SBSS  Objective  System,  an  Acquisition  Category  (ACAT) 
IC  Major  Defense  Acquisition  Program  (MDAP).7 

SBSS  Increment  1:  Pathfinder.  The  primary  goal  for  Increment  1 
of  the  SBSS  program  is  to  develop  a  single  space  vehicle,  the  SBSS 


7  As  noted  previously,  a  DoD  Space  MDAP  is  an  acquisition  program  that  is  designated  by 
either  the  DoD  Space  Milestone  Decision  Authority  or  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  as  being  of  special  interest  or  requiring  an  eventual 
total  expenditure  for  research,  development,  test,  and  evaluation  (RDT&E)  of  more  than 
$365  million  in  FY  2000  constant  dollars;  or,  for  procurement,  of  more  than  $2,190  billion 
in  FY  2000  constant  dollars  (Office  of  the  Under  Secretary  of  the  Air  Force,  2003). 
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Pathfinder  or  Block  10  satellite,  to  replace  the  SBV.  As  shown  in  Fig¬ 
ure  2.1,  the  threshold  requirement  for  the  Increment  1  system  is  sim¬ 
ply  to  replace  the  existing  but  threatened  SBV  capability  with  a  simi¬ 
lar  although  modestly  improved  capability.  Rapid  deployment,  rather 
than  dramatically  improved  capability,  is  the  highest  priority.  First 
launch  had  been  targeted  for  June  2007,  but  in  2005  the  launch  date 
was  rescheduled  for  2008.  In  late  2004,  a  $27  million  congressional 
cut  to  the  program’s  FY  2005  budget  forced  the  SBSS  SPO  to  delay 
the  preliminary  design  review,  which  had  been  scheduled  for  Octo¬ 
ber.  By  April  2005,  the  PDR  had  been  successfully  completed  (Boe¬ 
ing  Company,  2005). 

In  terms  of  cost  estimation  and  acquisition  management,  SBSS 
Increment  1  is  effectively  a  stand-alone  program.  FY  2004/2005  Bi¬ 
ennial  Budget  Estimates  for  the  Air  Force  indicate  that  the  approxi¬ 
mate  total  cost  through  FY  2009  for  the  Pathfinder  will  be  in  the 
$800  million  range,  whereas  Increment  2  is  expected  to  be  an  MDAP 
(Fernandez,  2004).  The  prime  contractor  for  the  Pathfinder  is  North¬ 
rop  Grumman  Mission  Systems;  in  March  2004,  Northrop  awarded 
Boeing  Integrated  Defense  Systems  a  $189  million  contract  to  de¬ 
velop  and  initially  operate  the  Pathfinder  (Morris,  2004).  A  Boeing 
press  release  suggested  that  follow-on  work  on  future  SBSS  satellites 
could  be  in  the  neighborhood  of  $2  billion  (Singer,  2004). 

In  addition  to  rapid  provision  of  an  urgently  needed  threshold 
capability,  the  SBSS  Increment  1  Pathfinder  is  intended  to  provide  a 
proof  of  concept  role  and  operational  feedback  for  evolving  and 
firming  up  the  requirements  for  the  objective  system  in  Increment  2. 
The  incremental  phasing  of  the  program  provides  the  opportunity  for 
feedback  of  requirements  for  the  objective  system,  as  shown  in  Figure 
2.2.  Specifically,  SBSS  Pathfinder  is  intended  to  make  the  following 
contributions  to  Increment  2: 
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Figure  2.1 

Increments,  Schedule,  and  Capability  Objectives  for  the  SBSS  Program 
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•  Define  the  initial  command  and  control  interfaces  for  future 
space  control  systems 

•  Break  ground  for  a  new  tasking,  processing,  exploiting,  and  dis¬ 
seminating  architecture 

•  Serve  as  a  tool  for  a  combined  task  force  to  develop  initial  tac¬ 
tics,  techniques,  and  procedures 

•  Serve  as  an  on-orbit  platform  for  characterizing  visible  sensor 
performance  limits. 

Given  this  program  structure  and  these  operational  and  devel¬ 
opmental  objectives  for  Increment  1,  SBSS  can  be  characterized  as 
using  a  spiral  development  implementation  strategy  (based  on  the 
definitions  provided  by  the  DoD  5000  guidance  documents),  since 
the  end  requirements  for  the  objective  system  are  not  known  at  pro¬ 
gram  inception,  but  rather  will  evolve  as  experience  is  gained  with  the 
threshold  system. 
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Figure  2.2 

Incremental  Phasing  Allows  Feedback  from  SBSS  Pathfinder  to 
Requirements  for  SBSS  Objective  System 
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SBSS  Increment  2:  The  Objective  System.  Notionally,  the  SBSS 
objective  system,  or  Block  20,  will  consist  of  a  four-  to  six-satellite 
constellation  in  low-earth  orbit,  and  will  emerge  from  the  second 
major  program  increment.  Its  broad  mission  objectives,  in  order  of 
priority,  are  to  do  the  following: 

•  Detect,  identify,  and  track  human-built  space  objects  in  a  timely 
manner 

•  Enable  global  strike  task  forces  to  find  and  track  deep-space  and 
near-earth  resident  space  objects 
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•  Support  the  space  surveillance  network  in  maintaining  an  accu¬ 
rate  catalog  of  all  resident  space  objects  (Space-Based  Space  Sur¬ 
veillance  Program  Office,  2004). 

The  more  specific  objective  system  requirements  for  Increment 
2  are  still  uncertain;  their  evolution  will  be  highly  dependent  on  the 
Pathfinder  experience.  Program  stretch-outs  will  provide  even  more 
time  for  feedback  from  Increment  1  into  Increment  2. 8  Initially,  Key 
Decision  Point  (KDP)  A — the  point  at  which  the  Increment  2  system 
will  move  into  the  concept  and  architecture  development  stage — was 
planned  for  FY  2006.  However,  the  SPO  now  expects  that  the  com¬ 
petition  to  select  a  prime  contractor  will  begin  sometime  in  FY  2007 
(Morris,  2004). 

As  a  result  of  the  spiral  development  approach  being  adopted, 
and  the  uncertainty  regarding  Increment  2  requirements,  the  SBSS 
SPO  emphasized  the  necessity  to  engage  in  evolutionary  cost  esti¬ 
mating  as  the  program  evolves  as  the  only  feasible  approach  to 
achieving  high-quality  cost  estimates.  According  to  the  SPO,  the 
evolutionary  costing  approach  has  posed  few  challenges,  and  is  con¬ 
sidered  to  be  functioning  well.  Since  the  program  is  not  now  an 
MDAP,  and  is  relatively  low-profile,  uncertainties  over  ultimate  cost 
that  arise  from  evolutionary  costing  do  not  seem  to  have  provoked 
any  political  reactions.  This  is  not  true,  however,  with  larger,  higher- 
profile  programs,  as  discussed  later  in  this  chapter. 

RAIDRS:  EA  as  Potential  Foot  in  the  Door? 

Introduction.  RAIDRS  as  currently  structured  is  an  even  more 
modest,  relatively  low-cost,  low-risk  program  compared  to  SBSS. 
Also  like  SBSS,  the  first  segment  of  RAIDRS  is  in  effect  structured  as, 


8  In  July  2004,  Congress  cut  $27  million  from  the  Air  Force’s  SBSS  request  of  $102  million 
for  FY  2005.  This  cut  was  reportedly  due  to  congressional  frustration  over  a  six-month  delay 
in  awarding  $189  million  to  a  Boeing-Ball  Aerospace  team  subcontracted  to  Northrop.  Ac¬ 
cording  to  the  SBSS  SPO,  the  contract  delay  was  due  to  unspecified  “launch  risks.”  The  SPO 
claimed  that  the  funding  cut  could  delay  the  launch  (scheduled  for  2007),  but  would  not 
affect  program  capability.  The  launch  date  has  now  moved  to  2008  (Collard-Wexler  et  al., 
2004). 
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and  is  being  treated  as,  a  stand-alone  program.  In  its  current  form, 
RAIDRS  is  essentially  a  software  and  ground  equipment  integration 
effort,  with  no  dedicated  new  SV  involved.  Unlike  SBSS,  however, 
the  current  RAIDRS  program  originated  from  a  larger  and  more  am¬ 
bitious  earlier  planned  version  of  the  program  informally  known  as 
“Big  RAIDRS.”  On  the  positive  side,  EA  was  adopted  for  RAIDRS  to 
break  the  program  up  into  more  manageable,  lower-risk  increments, 
to  provide  an  earlier  operationally  useful  capability,  and  to  fit  the 
program  into  current  budget  realities.  Yet  some  program  officials  ex¬ 
pressed  concerns  to  us  that  some  contractors  may  view  the  adoption 
of  an  EA  strategy  as  a  ploy  to  keep  the  Big  RAIDRS  concept  alive 
during  a  period  of  budgetary  difficulties  and  uncertainties  over  the 
ultimate  objective  requirements  for  the  program.  A  possible  negative 
implication  is  that  EA  is  being  perceived  by  some  as  a  means  to  keep 
a  foot  in  the  door  for  Big  RAIDRS,  thus  potentially  leading  to  the 
commitment  of  the  government  to  a  program  that  may  end  up  cost¬ 
ing  much  more  than  currently  projected. 

Big  RAIDRS  floundered  in  the  2000-2003  time  frame  because 
of  difficulties  over  requirements  definition  and  budgetary  constraints. 
Big  RAIDRS  illustrates  some  of  the  budgetary  challenges  that  can  be 
encountered  on  large,  high-visibility  programs  that  adopt  a  spiral  de¬ 
velopment  approach.  When  final  objective  requirements  remain 
vague  and  ill  defined  (as  they  will  always  be  at  the  beginning  of  a  spi¬ 
ral  development  effort),  overall  program  cost  estimation  is  difficult. 
In  potentially  large-scale  high-cost  programs  such  as  Big  RAIDRS, 
the  senior  political  and  service  leadership  may  be  uncomfortable  with 
the  attendant  program  cost  uncertainties,  particularly  in  light  of  the 
Young  Panel’s  strong  criticism  of  historic  cost  growth  on  past  space 
programs,  and  the  NSSAP  03-01  emphasis  on  cost  estimating  real¬ 
ism.  Not  surprisingly,  then,  Big  RAIDRS  evolved  toward  EA,  and 
eventually  adopted  an  incremental  rather  than  a  spiral  development 
approach,  at  least  in  principle.  However,  the  immediate,  near-term 
reason  for  adopting  an  EA  approach  emerged  in  2003  when  an  ur¬ 
gent  need  arose  for  very  rapid  fielding  of  an  initial  operationally  use- 
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ful  capability.  In  this  sense,  RAIDRS  Spiral  1  is  very  similar  in  origin 
and  structure  to  SBSS  Increment  l.9  For  this  type  of  situation,  where 
rapid  deployment  of  a  basic  threshold  capability  is  considered  urgent, 
EA  strategies  appear  to  be  well  suited. 

The  RAIDRS  program  had  been  originally  conceptualized  as  a 
multi-billion  dollar  program  to  meet  two  missions:  Defensive  Coun¬ 
terspace  (DCS)  and  Space  Situational  Awareness  (SSA).  DCS  activi¬ 
ties  consist  of  “active  and  passive  actions  to  protect  our  space-related 
capabilities  from  enemy  attack  or  interference,”  while  SSA  is  the 
“ability  to  know  the  exact  whereabouts  and  actions  of  Earth-orbiting 
objects”  (U.S.  Air  Force,  1997,  p.  48;  2001,  p.  14). 

As  noted  earlier,  it  proved  very  difficult  to  settle  on  an  overall 
consensus  concept  for  the  original  Big  RAIDRS  program — especially 
for  its  DCS  mission — and  after  about  three  years  of  conceptual  de¬ 
velopment,  it  did  not  appear  that  the  necessary  budget  for  achieving 
it  would  be  forthcoming.  But  hostile  activities  against  U.S.  military 
satellite  communications  in  early  2002  brought  a  new  level  of  ur¬ 
gency  to  the  requirement  to  protect  them.  Therefore,  in  2003,  the 
Big  RAIDRS  program  was  restructured  in  both  concept  and  pro¬ 
posed  budget  and  transformed  into  a  new  RAIDRS  program  with  a 
more  limited,  but  much  clearer,  near-term  threshold  objective:  to  de¬ 
velop  and  quickly  make  operational  a  system  that  could  differentiate 
hostile  attacks  on  military  satellite  communications  from  natural  or 
unintentional  events,  and  enable  the  timely  deployment  of  defensive 
responses  to  counter  threats  (Rapid  Attack  Identification,  Detection, 
and  Reporting  System  Program  Office,  2004). 

RAIDRS  Spiral  1:  Fast-Track  Threshold  Capability.  RAIDRS  pro¬ 
gram  management  decided  that,  given  restricted  development  dollars 
and  the  new  and  pressing  need  for  a  rapidly  fielded  capability,  it 


9  Note  the  technically  incorrect  use  of  the  term  “spiral”  to  describe  the  first  development 
element  of  the  RAIDRS  program.  Several  other  programs  examined  appear  to  apply  EA 
terminology  incorrectly,  particularly  the  terms  “spiral”  and  “incremental.”  Some  programs 
have  changed  the  terminology  they  use  over  time.  This  indicates  persistent  confusion  over 
the  new  DoD  definitions,  or  at  least  a  lack  of  precision  in  their  use.  It  appears  that  the  trend 
toward  using  the  term  “increment”  rather  than  “spiral”  also  reflects  a  general  trend  away 
from  the  formal  use  of  the  spiral  development  approach,  as  discussed  elsewhere  in  this  book. 
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made  sense  to  recast  RAIDRS  as  an  evolutionary  acquisition  pro¬ 
gram.  Both  budget  and  schedule  considerations  have  led  managers  to 
design  a  RAIDRS  Spiral  1  system  that  does  the  following: 

•  Relies  heavily  on  available  commercial  and  government  hard¬ 
ware 

•  Protects  only  the  most  vulnerable  communication  satellites 

•  Leverages  existing  space  assets. 

A  formal  request  for  proposals  for  RAIDRS  Spiral  1  was  released 
on  October  22,  2004,  with  a  November  2004  deadline  for  submis¬ 
sions.  Like  SBSS  Increment  1,  RAIDRS  Spiral  1  is  a  relatively  modest 
effort.  In  2004,  RAIDRS  program  managers  estimated  that  the 
budget  for  RAIDRS  Spiral  1  will  come  to  roughly  $15  million  per 
year  in  FY  2005  and  2006,  rising  to  approximately  $35  million  at 
IOC  in  FY  2007. 10  In  total,  RAIDRS  was  expected  to  cost  about 
$136  million  through  full  operational  capability  (FOC),  which  was 
targeted  for  FY  2009.  To  control  costs,  the  Spiral  1  system  hardware 
will  be  acquired  as  commercial  off-the-shelf  (COTS),  government 
off-the-shelf  (COTS),  or  build- to-print,  with  roughly  50  percent  of 
the  hardware  expected  to  be  COTS.  Much  of  the  RAIDRS  Spiral  1 
effort  is  devoted  to  developing  software  that  will  allow  ground  con¬ 
trollers  to  manipulate  and  interpret  data  downloaded  from  satellites. 

RAIDRS’  focus  at  Spiral  1  initial  operational  capability  (IOC) 
will  be  on  the  Defense  Satellite  Communications  System  (DSCS). 
Focusing  on  DSCS  will  also  allow  RAIDRS  to  leverage  capabilities 
provided  by  the  U.S.  Army’s  interference  monitoring  power  control 
subsystem  (IMPCS),  which  already  monitors  DSCS  for  electromag¬ 
netic  interference,  but  can  only  determine  the  source  of  the  interfer¬ 
ence  after  communications  have  been  processed.*  11  RAIDRS  will  use 
information  provided  by  IMPCS  to  geolocate  the  source  of  interfer- 


10  Congress  approved  an  FY  2005  budget  of  $16.4  million  for  RAIDRS  in  July  2004. 

11  IMPCS  does  not  distinguish  between  interference  due  to  hostile  attacks  versus  natural 
events.  The  Army  uses  it  simply  to  reallocate  bandwidth  away  from  satellites  experiencing 
interference. 
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ence  in  transponded  communications  and  report  it  to  the  Air  Force 
in  near  real  time.  One  suite  of  equipment  (a  single  interference  detec¬ 
tion  system  [IDS]  unit  plus  a  geolocation  unit)  will  be  required  to 
achieve  these  initial  capabilities. 

At  FOC,  RAIDRS  Spiral  1  is  projected  to  have  more  units  than 
at  IOC — perhaps  as  many  as  eight  geolocation  units,  36  interference 
detection  system  units,  and  a  laser  dazzling  detection  unit.  These  ad¬ 
ditional  units  will  allow  it  to  monitor  all  critical  communications 
worldwide,  detect  electromagnetic  interference  with  satellite  commu¬ 
nications  and  laser  dazzling  of  Defense  Support  Program  (DSP)  early- 
warning  system  satellites,  geolocate  attackers,  and  report  the  event  to 
the  designated  Air  Force  Operations  Center  (AOC)  for  space.  FOC 
for  the  objective  requirement  for  Spiral  1  is  planned  for  FY  2009,  but 
this  will  be  driven  more  by  budgetary  than  by  technological  consid¬ 
erations  because  the  added  capabilities  of  the  objective  system  mostly 
do  not  involve  development  of  new  technology. 

In  May  2005,  the  Air  Force  awarded  the  RAIDRS  Spiral  1  con¬ 
tract  to  Integral  Systems,  a  major  provider  of  satellite  ground  systems. 
As  prime  contractor,  Integral  Systems  teamed  with  numerous  other 
contractors,  including  Intelsat,  Northrop  Grumman,  and  QinetiQ. 
The  contract  was  for  the  relatively  modest  amount  of  $123  million 
over  five  years.  Integral  Systems’  technical  proposal  relies  heavily  on 
COTS  equipment.  Earlier,  Integral  Systems  also  won  a  study  contract 
to  investigate  concepts  and  technologies  further  for  RAIDRS  Spiral  2 
(Integral  Systems,  2005). 

RAIDRS  Spiral  2:  Keeping  the  Door  Open  for  Big  RAIDRS?  Be¬ 
cause  the  Spiral  1  system  will  have  only  a  command  and  control  func¬ 
tion  (allowing  users  to  geolocate  the  source  of  electromagnetic  inter¬ 
ference  but  not  to  assess  the  nature  of  the  threat),  one  concept  for 
follow-on  spirals,  if  they  are  undertaken,  is  the  development  of  a  deci¬ 
sion  support  element.  This  element  would  be  able  to  read  informa¬ 
tion  provided  by  the  RAIDRS  interference  detection  and  geolocation 
systems  and  automatically  combine  it  with  data  from  intelligence  and 
other  databases  to  assess  the  seriousness  of  the  threat.  Alternatively, 
future  spirals  could  simply  involve  an  increase  in  the  number  of  units 
developed  in  Spiral  1.  Yet  through  most  of  2004,  the  requirement 
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and  even  the  concept  of  a  possible  Spiral  2  remained  largely  un¬ 
known. 

As  of  late  2004,  there  was  little  funding  available  to  support  ad¬ 
ditional  concept  definition  or  requirements  development  for 
RAIDRS  Spiral  2.  Furthermore,  the  compressed  Spiral  1  schedule 
permits  only  limited  feedback  from  Spiral  1  into  the  requirements 
generation  process  for  follow-on  spirals.  In  2004,  the  SPO  began  to 
prepare  program  research  and  development  documents  (PRDAs)  for 
Spiral  2,  and  requested  industry  feedback  on  a  statement  of  objectives 
for  Spiral  2  by  October  6,  2004.  At  that  time,  the  SPO  expected  to 
spend  between  $5  million  and  $10  million  by  FY  2007  to  refine  sys¬ 
tem  requirements  for  Spiral  2.  It  hopes  to  achieve  KDP  B  (approval 
of  concept  development)  for  Spiral  2  by  FY  2008.  In  late  2004,  how¬ 
ever,  this  budget  and  schedule  continued  to  be  largely  notional.  Thus, 
in  effect,  RAIDRS  Spiral  1  remained  essentially  a  stand-alone  pro¬ 
gram.  During  FY  2004,  nearly  all  of  the  requirements  documents, 
trade  studies,  and  cost  estimates  continued  to  focus  on  Spiral  1 . 

During  2004,  SPO  officials  believed  it  was  quite  possible  that 
Spiral  1  would  conclude  the  RAIDRS  program:  The  contracts  for 
Spirals  1  and  2  are  entirely  separate,  and  it  is  not  clear  that  contrac¬ 
tors’  work  experience  and  success  on  Spiral  1  will  necessarily  increase 
the  probability  of  winning  contracts  for  Spiral  2.  As  noted  previously, 
Integral  Systems  was  also  awarded  a  Spiral  2  study  contract.  The  ini¬ 
tial  funding,  however,  was  very  small.  The  February  2005  Air  Force 
budget  justification  projected  expenditure  of  only  $5.7  million 
through  FY  2006  on  Spiral  2  concept  definition  and  pre-acquisition 
architecture  and  system  development.  This  number  was  projected  to 
rise  by  $7.1  million  in  FY  2007  (U.S.  Department  of  the  Air  Force, 
2005,  p.  962). 

According  to  the  SPO,  at  least  some  contractors  apparently  be¬ 
lieve  that  Spiral  1  could  be  an  entree  to  a  much  larger  reborn  Big 
RAIDRS  program  in  the  future.  RAIDRS  Spiral  1  program  managers 
raised  some  concerns  that,  to  the  extent  contractors  are  correct,  any 
such  Big  RAIDRS  program  may  be  susceptible  to  requirements  creep, 
a  problem  that  bedeviled  the  original  Big  RAIDRS  program.  In  the 
view  of  some  program  officials,  acquisition  reform  measures  have 
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transferred  so  much  influence  over  system  specifications  to  contrac¬ 
tors,  that  in  a  program  with  ill-defined  requirements,  requirements 
creep  may  become  a  significant  challenge. 

Finally,  with  respect  to  the  program  cost  analysis  effort,  the 
costing  of  RAIDRS  Spiral  1  and  Spiral  2  has  also  been  kept  entirely 
separate.  While  the  costing  of  Spiral  1  certainly  involves  challenges, 
its  boundaries  are  fairly  well  defined,  because  the  requirement  is  rea¬ 
sonably  clear  and  unambiguous.  Even  for  Spiral  1,  the  SPO  has  en¬ 
gaged  in  an  evolutionary  costing  effort  as  the  content  of  the  spiral  has 
evolved.  The  situation  is  totally  different  for  Spiral  2,  however.  With 
no  system  concept,  concept  of  operations,  or  objective  requirements 
clearly  defined,  it  is  simply  not  possible,  in  the  view  of  the  SPO,  to 
do  any  sort  of  meaningful  cost  estimates  for  the  Spiral  2  system.  This 
has  led  some  observers  to  question  whether  RAIDRS  Spiral  1  might 
be  viewed  as  a  low-ball  buy-in  that  masks  a  commitment  to  a  much 
larger  program.  As  of  late  2004,  the  relevance  of  such  a  criticism 
could  not  be  assessed,  but  clearly  the  SPO  expressed  concerns  that  at 
least  some  contractors  might  view  the  program  this  way. 

GPS  III:  Prudent  Incrementalism  or  a  Foot  in  the  Door? 

Introduction.  Unlike  the  previous  two  systems,  Global  Posi¬ 
tioning  Satellite  (GPS)  III  is  a  follow-on  program  to  a  well- 
established  and  highly  successful  existing  program.  Nonetheless,  GPS 
III  is  a  very  ambitious  large-scale  space  acquisition  program  envi¬ 
sioning  major  investments  in  the  development  and  procurement  of  a 
series  of  totally  new  space  vehicles  (SVs),  ground  control  hardware, 
and  receiver  units.  The  Air  Force  expects  to  commit  $830  million  to 
the  GPS  III  program  through  FY  2011,  with  first  launch  of  a  new  SV 
expected  in  FY  2013  (U.S.  Department  of  the  Air  Force,  2005,  p. 
619).  In  2004,  GPS  III  was  still  in  a  very  early  system  definition 
stage.  Program  managers  initiated  plans  for  a  long,  potentially  rather 
complex  multiphase  EA  approach  that  notionally  aimed  at  achieving 
an  initial  launch  capability  (ILC)  in  FY  2012  to  attain  a  threshold 
capability.  At  least  four  notional  spirals  (called  Increments  A-D)  were 
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under  consideration  and  being  assessed  in  2004. 12  The  initial  incre¬ 
ment  (Increment  A)  was  intended  to  meet  a  threshold  capability, 
while  Increment  D  was  aimed  at  achieving  the  objective  system  capa¬ 
bility. 

The  GPS  system  of  course  has  been  in  place  for  decades.  The 
existing  NAVSTAR  Global  Positioning  System  dates  back  originally 
to  the  1970s.  Currently,  it  is  made  up  of  a  24-satellite  constellation 
providing  24-hour  navigation  and  timing  information  to  military  and 
civilian  users  worldwide.  GPS  satellites,  in  one  of  six  medium  earth 
orbits,  circle  the  earth  every  12  hours  emitting  continuous  navigation 
signals  on  two  L-band  frequencies,  LI  and  L2.  In  addition  to  the 
space  segment,  the  GPS  system  consists  of  a  worldwide  satellite  con¬ 
trol  network  (the  control  segment)  and  GPS  receiver  units  (the  user 
segment)  that  acquire  the  satellite’s  signals  and  translate  them  into 
precise  position  and  timing  information. 

There  have  been  five  major  block  upgrades  to  the  GPS  space 
segment  since  its  initial  launch  during  the  1970s:  Block  II,  Block  IIA, 
Block  HR,  Block  IIR-M,  and  Block  II-F.  In  1997,  Block  IIRs  began 
replacing  Block  II  and  IIAs,  offering  a  precise  positioning  service  on 
LI  and  L2  and  20  MHz  bandwidth.  The  launch  of  the  first  Block 
IIR-M,  offering  a  second  civil  signal  on  L2  plus  a  new  military-code 
signal  on  LI  and  L2,  took  place  in  September  2005.  The  next  up¬ 
grade  of  GPS,  the  Block  II-F,  providing  a  third  civil-only  signal  on 
L5  and  boasting  a  longer  design  life  than  previous  models,  is  sched¬ 
uled  to  launch  in  2007. 

According  to  officials  within  the  GPS  III  Joint  Program  Office 
(JPO),  each  of  the  GPS  Block  II  upgrades  has  involved  mainly  soft¬ 
ware  changes  combined  with  relatively  modest  changes  to  system 
hardware,  in  part,  because  the  primary  objective  requirement  for  GPS 
II,  improved  accuracy,  has  remained  the  same  over  time.  Further,  all 
of  the  Block  II  satellites  have  essentially  been  passive,  transmitting 
their  signals  indiscriminately  and  without  charge  to  all  users. 


12  Once  again,  note  the  mixing  and  incorrect  use  of  terms  “spirals”  and  “increments,”  which, 
technically  speaking,  violates  the  definitions  in  the  DoD  5000  series  guidance. 
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The  vision  for  GPS  III  is  quite  different,  involving  a  broad  range 
of  possible  new  missions  and  new  objective  end  capabilities  for  the 
overall  system.  Most  important  among  the  new  military  capabilities 
envisioned  are  much  more  robust  navigational  warfare  (NAVWAR) 
capabilities,13  significantly  improved  accuracy,  near  real-time  con¬ 
stellation  management,  and  signal  flexibility  and  reprogrammability. 
These  in  turn  require  the  progressive  development  and  upgrading  of  a 
series  of  entirely  new  space  vehicles  (SVs)  and  ground  support  hard¬ 
ware,  along  with  new  software.  For  these  reasons,  program  managers 
argue,  acquisition  of  the  GPS  III  system  is  particularly  well  suited  to 
an  evolutionary  approach.  As  noted  previously,  the  GPS  III  program, 
as  of  late  2004,  notionally  consisted  of  four  developmental  spirals, 
culminating  in  the  objective  system. 

GPS  III  Spiral  1:  Meeting  Threshold  Requirements.  As  of  mid- 
2004,  GPS  III  Spiral  1  had  already  evolved  considerably  in  its  plan¬ 
ning  and  structural  maturity.  Spiral  1  was  expected  to  add  the  fol¬ 
lowing  broad  capabilities  to  those  already  existing  on  the  GPS  II-F 
variant: 

•  A  dual-launch  capability 

•  A  satellite  vehicle  bus  designed  for  future  technology  insertion 

•  Improved  accuracy 

•  Some  ability  to  deny  access  to  enemies 

•  Increased  signal  flexibility  and  jam  resistance. 

As  of  2004,  program  funding  requirements  were  put  at  $2.1  bil¬ 
lion  through  FY  2009;  the  development  contract  was  scheduled  to  be 
awarded  in  2006,  with  initial  launch  capability  (ILC)  expected  in  FY 
2012.  Both  the  budget  and  schedule  may  still  undergo  significant 
changes,  however,  as  Spiral  1  is  still  in  acquisition  Phase  A,  the  con- 


13  A  key  aspect  of  the  improved  NAVWAR  capability  is  based  on  the  spot  beam  concept 
that  is  intended  to  permit  around-the-clock  military  operations  in  heavily  jammed  environ¬ 
ments  with  no  mobility  constraints. 
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cept  development  phase.14  The  final  product  of  this  phase  will  be  a 
system  specification  for  a  satellite  bus  to  be  developed  by  contractors 
in  Phase  B. 

In  a  significant  departure  from  previous  GPS  programs,  a  key 
feature  of  the  evolutionary  GPS  III  program  is  that  GPS  III  Spiral  I 
involves  parallel  and  harmonized  development  of  the  user  equipment 
and  ground  control  segments.  In  fact,  both  the  capability  develop¬ 
ment  document  (CDD)  and  concept  of  operations  (CONOPS) 
document  for  GPS  III  explicitly  require  complete  ground  segment 
control  functionality  by  the  end  of  Spiral  1 .  According  to  program 
managers,  the  strictness  of  this  phasing  requirement  is  due  to  past 
problems  with  space  acquisition  programs,  where  the  ground  control 
segment  development  was  not  adequately  coordinated  with  the  SV 
element,  thus  blocking  the  ability  to  take  full  advantage  of  space  seg¬ 
ment  capability  after  ILC. 

GPS  III  Future  Spirals:  Defining  a  Path  to  Objective  Capability. 

The  GPS  III  program  is  in  its  very  earliest  concept  development  and 
planning  stages.  The  JPO  has  adopted  a  competitive  approach  with 
multiple  contractors  to  maturing  GPS  III  requirements  for  a  formal 
system  requirements  review  (SRR)  in  2005  and  to  support  the  KDP 
B  review.  A  full  and  open  competition  to  award  a  single  GPS  III  de¬ 
velopment  contract  is  planned  for  FY  2006  to  support  the  SDR.  The 
draft  CDD  for  GPS  III  identifies  the  threshold  and  objective  re¬ 
quirements  for  the  full  system.  As  of  early  2005,  the  SDR  and  KDP- 
B  were  planned  for  2007  (U.S.  Department  of  the  Air  Force,  2005, 
p.  624). 

The  Spiral  1  system,  of  course,  is  being  designed  to  meet  the 
threshold  requirements.  However,  as  of  2004,  development  of  CDDs 
for  follow-on  spirals,  which  will  define  the  path  to  objective  capabil¬ 
ity,  are  not  planned  for  another  three  to  four  years.  Two  competing 
contractors  (Boeing  and  Lockheed)  are  generating  their  own  unique 
company-specific  content  for  potential  paths  to  objective  capability. 
Individual  steps  along  the  path  were  originally  defined  in  terms  of 


14  According  to  GPS  III  program  managers,  Phase  A  was  initiated  in  FY  2000,  but  has  been 
disrupted  three  times  due  to  budget  instability. 
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technical  requirements,  but  they  were  later  redefined  in  terms  of  ca¬ 
pabilities,  leaving  the  determination  of  the  technical  means  of 
achieving  those  capabilities  to  the  contractors.  As  of  mid-2004,  very 
large  uncertainties  remained  regarding  the  final  structuring  and  tech¬ 
nical  content  of  the  follow-on  phases,  and  the  precise  path  to  full  ob¬ 
jective  capabilities. 

Cost  estimating  on  the  GPS  III  program  is  being  approached  in 
much  the  same  way  as  on  the  RAIDRS  (and  SBSS)  program:  Spiral  1 
is  being  costed  through  FY  2011,  based  on  the  evolving  threshold  re¬ 
quirements,  while  future  spirals  will  be  costed  as  entirely  separate 
programs.  Yet,  as  of  mid-2004,  so  few  specifics  had  been  worked  out 
regarding  follow-on  EA  phases  that  no  meaningful  costing  effort  was 
possible  beyond  Spiral  1.  Therefore,  as  with  RAIDRS,  there  have 
been  questions  raised  in  some  quarters  as  to  whether  the  GPS  III  ac¬ 
quisition  strategy  represents  wise  incrementalism,  or,  rather,  could  be 
characterized  as  a  foot  in  the  door  for  a  much  larger  program  that  has 
not  yet  been  openly  acknowledged.  Yet  the  GPS  III  program  has  pro¬ 
voked  relatively  little  controversy,  in  part  because  of  the  long  histori¬ 
cal  track  record  of  success  for  GPS,  and  because  of  the  widespread, 
incorrect  impression  that  GPS  III  merely  represents  another  series  of 
block  upgrades  on  the  same  pattern  as  the  current  GPS  II  system. 

SBR:  A  Large,  Complex,  High-Profile  EA  Program 

Introduction.  The  Space-Based  Radar  (SBR)  is  a  much  larger, 
more  complex,  and  higher  profile  EA  program  than  the  others  re¬ 
viewed  here.15  It  envisions  developing  a  range  of  far-reaching  and 
very  ambitious  space-based  surveillance  capabilities  to  serve  multiple 
user  communities.  It  is  a  potentially  very  demanding  program,  both 
technologically  and  organizationally,  which  must  satisfy  the  diverse 
needs  and  preferences  of  many  very  different  user  communities.  Be¬ 
cause  of  these  characteristics,  major  challenges  in  the  early  phases  of 
the  program  have  included  the  issues  of  defining  both  threshold  and 
objective  requirements  and  capabilities  (both  for  the  overall  program 


15  The  SBR  program  was  restructured  in  2005  and  renamed  the  Space  Radar  (SR)  program. 


Evolutionary  Acquisition  in  Practice  67 


and  within  increments);  selecting  the  appropriate  technologies  and 
capabilities,  and  the  phasing  of  their  insertion;  and  the  structuring  of 
the  program  EA  increments  and  the  precise  path  to  the  objective  end 
capabilities  of  the  system.  In  addition,  considerable  debate  and  dis¬ 
cussion  have  surrounded  the  question  of  the  cost  effectiveness  of  the 
capabilities  provided  by  program,  the  credibility  of  various  official 
and  unofficial  cost  estimates  of  the  program,  and  the  budgetary 
phasing  of  the  effort. 

The  SBR  program  began  in  2001  following  the  cancellation  of 
the  Discoverer  II  program  by  Congress  due  to  concerns  over  cost, 
schedule  risk,  and  uncertainty  over  requirements.  During  its  relatively 
brief  tenure  in  its  original  form  (from  2001  to  2005),  SBR  also  expe¬ 
rienced  considerable  controversy  over  similar  issues  that  was  reflected 
in  major  funding  instabilities.  For  the  purposes  of  this  monograph, 
however,  the  authors  primarily  focus  on  only  one  aspect  of  the  pro¬ 
gram:  how  program  officials  attempted  to  apply  EA  concepts  to  the 
SBR  acquisition  approach  to  help  reduce  cost  and  technological  risk 
and  achieve  a  more  rapid  fielding  of  an  operationally  useful  capabil¬ 
ity.  The  SBR  experience  suggests  that  in  such  a  complex  and  high- 
profile  program  as  this,  the  successful  application  of  a  coherent  EA 
strategy  is  enormously  challenging.  Nonetheless,  despite  many  bumps 
in  the  road,  senior  DoD  Space  leadership  has  continued  to  support 
SBR  and  is  heavily  committed  to  using  a  form  of  EA  to  move  the 
program  forward  while  controlling  risks  and  costs.  One  could  even 
argue  that  the  program  restructuring  into  a  technology  demonstration 
effort  in  2005  represented  a  legitimate  attempt  to  apply  EA  concepts 
more  rigorously  to  a  large  and  complex  developmental  effort. 

As  noted  previously,  the  SBR  program  grew  out  of  Discoverer 
II,  a  joint  Air  Force,  Defense  Advanced  Research  Projects  Agency 
(DARPA),  and  National  Reconnaissance  Office  (NRO)  technology 
demonstration  project  that  was  intended  to  improve  the  coverage  and 
timeliness  of  space-based  reconnaissance  and  surveillance.  Congress 
canceled  Discoverer  II  in  2000,  but  approved  $30  million  for  the 
NRO  to  continue  further  technical  development  for  the  space-based 
radar  concept.  According  to  press  accounts,  Congress  cancelled  Dis¬ 
coverer  II  because  of  “uncertain  costs  and  schedule,  poorly  explained 
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requirements,  and  a  lack  of  coherent  vision  for  how  the  system  would 
transition  to  operational  use”  (Tirpak,  2002,  p.  64).  Similar  criticisms 
would  continue  to  dog  the  follow-on  SBR  program  effort. 

When  the  Air  Force  became  the  executive  agent  for  military 
space  activities  in  2001,  it  took  over  leadership  of  the  joint  Air 
Force-NRO  program.  The  SBR  program  was  initiated  in  November 
2001  to  provide  military  forces  with  the  capability  to  find,  identify, 
track,  and  monitor  moving  targets  on  the  ground  from  space.  The 
central  mission  for  SBR  was  extremely  demanding:  to  provide  persis¬ 
tent  global  surveillance  and  global  situational  awareness  using  ad¬ 
vanced  technologies  for  surface  moving  target  indication  (SMTI), 
synthetic  aperture  radar  (SAR)  imaging,  and  high  resolution  terrain 
information  (HRTI). 

One  of  the  most  challenging  aspects  of  the  SBR  program  was 
the  need  to  harmonize  military  and  intelligence  requirements  for 
processing  and  disseminating  data  on  both  mobile  and  stationary  tar¬ 
gets  from  around  the  globe  to  multiple  diverse  user  communities  with 
sometimes  differing  objectives.  As  one  observer  wryly  noted,  with 
perhaps  a  bit  of  exaggeration,  there  were  somewhere  between  “fifteen 
and  eighteen  customers  who  all  want  different  things”  from  the  SBR 
system.  Daily  management  of  the  program  was  complicated  by  the 
existence  of  a  JPO  that  represented  a  range  of  organizations,  both  in¬ 
side  and  outside  of  the  Air  Force,  which  were  often  advancing  varying 
and  sometimes  conflicting  agendas.  Inside  the  Air  Force  and  on  the 
acquisition  side,  these  included  both  the  Space  and  Missile  Systems 
Center  (SMC),  and  the  Electronic  Systems  Center — with  the  latter 
focusing  on  SBR  program  aspects  related  to  Battlefield  Management 
and  Command,  Control,  and  Communications  (BMC3).  Other 
major  players  at  the  SBR  JPO  include  the  NRO,  the  Army,  the  Navy, 
and  the  National  Geospatial  Intelligence  Agency  (NGA). 

In  the  early  days  of  the  program,  disagreements  among  various 
potential  stakeholders  and  user  groups  regarding  mission  priorities  led 
to  disputes  over  fundamental  system  architectural  issues  and  program 
phasing.  For  example,  if  greatest  priority  is  placed  on  achieving  an 
early  high  capability  in  wide-area  SMTI,  then  a  higher  altitude  con¬ 
stellation  (medium  earth  orbit  or  MEO  constellation)  of  SVs  with 
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lower  resolution  capabilities  would  be  selected  as  the  first  priority. 
However,  if  the  greatest  emphasis  is  placed  on  the  intelligence  gath¬ 
ering  missions  aimed  at  fixed  installations,  then  preference  might 
shift  to  a  lower  altitude  constellation  (low  earth  orbit  or  LEO  con¬ 
stellation)  with  higher  resolution  capabilities.  The  selection  of  one 
architecture  over  another  has  enormous  implications  for  the  number 
of  SVs  that  might  need  to  be  procured,  the  type  of  SVs,  and  the  over¬ 
all  cost  of  the  program.  Since  the  senior  DoD  leadership  determined 
early  in  the  program  that  only  one  overall  system  was  affordable  for 
both  intelligence  and  warfighting  purposes,  SBR  had  to  be  a  com¬ 
promise  system  that  met  both  the  intelligence  and  warfighting  com¬ 
munities’  requirements,  and  thus  had  to  contain  both  MEO  and 
LEO  elements.  However,  the  exact  mix  was  obviously  open  to  debate. 

In  addition  to  these  types  of  difficult  challenges  regarding  the 
definition  of  system  architectures,  capabilities,  and  requirements  for 
multiple  users  and  arbitrating  among  legitimately  differing  needs,  the 
SBR  program  was  also  complicated  by  the  advocacy  of  alternative 
technical  solutions  and  architectures,  and  by  organizational  turf  wars 
over  which  entity  would  act  as  the  ultimate  control  authority  for  the 
overall  system  and  its  requirements.  Here  again,  the  most  challenging 
issues  arose  from  the  differing  perspectives  of  the  intelligence  and  op¬ 
erational  warfighting  communities.  In  addition,  considerable  discus¬ 
sion  also  emerged  over  the  optimal  mix  between  and  integration 
among  space-based,  land-based,  and  air  breathing  assets  such  as  Joint 
Surveillance  Target  Attack  Radar  System  (JSTARS)  and  unmanned 
aerial  vehicles  (UAVs)  platforms. 

Finally,  to  complicate  program  management  and  cost  estimation 
efforts  further,  the  program  was  increasingly  subjected  to  intense  and 
sometimes  hostile  public  scrutiny  and  skepticism  from  Congress  and 
other  outside  observers,  for  a  variety  of  reasons.  There  is  no  denying 
that,  no  matter  how  it  is  defined,  SBR  was  likely  to  be  a  very  expen¬ 
sive  program.  As  of  early  2005,  program  costs  at  the  very  minimum 
were  officially  projected  at  $28  billion  through  FY  2024  (Rees, 
2005).  Early  in  the  life  of  the  program,  some  estimates  as  high  as  $68 
billion  for  some  types  of  possible  architectures  were  circulated  in  the 
defense  press  and  in  the  halls  of  Congress.  Much  lower  independent 
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cost  estimates  in  the  range  of  $34  billion  in  FY  2004  dollars  were 
produced  by  the  DoD  Cost  Analysis  Improvement  Group  (CAIG) 
and  by  the  Program  Office.16  Nonetheless,  even  the  lower  estimates 
meant  that  SBR  would  be  one  of  the  largest  and  costliest  space  pro¬ 
grams  ever  in  the  Air  Force  portfolio.  In  the  wake  of  the  Young  Panel 
findings  and  the  severe  criticisms  of  large  cost  overruns  and  poor  cost 
estimating  track  records  on  such  programs  as  the  Space-Based  Infra¬ 
red  System  High  (SBIRS  High)  and  other  space  acquisition  efforts,  it 
is  not  surprising  that  SBR  was  increasingly  subjected  to  a  healthy  dose 
of  scrutiny  and  skepticism.17 

The  SBR  JPO  implemented  numerous  management  initiatives 
in  response  to  these  circumstances.  To  ensure  requirements  harmoni¬ 
zation  among  the  diverse  user  communities,  and  to  control  require¬ 
ments  creep  and  unwarranted  cost  growth,  the  SBR  program  office 
established  three  senior-level  requirements  oversight  groups  consisting 
of  senior  officials  from  OSD,  the  military  services,  and  the  intelli¬ 
gence  community.  Each  of  these  groups  was  responsible  for  advising 
the  joint  program  office  on  the  acceptability  of  various  cost,  schedule, 
and  performance  tradeoffs.  Another  major  initiative  was  an  extensive 


16  The  SBR  JPO  argued  that  the  high  estimates  were  utterly  flawed  because  they  were  not 
founded  on  the  official  baseline  assumptions  established  by  DoD.  Some  critics  in  Congress 
and  elsewhere,  however,  were  skeptical  of  the  CAIG  and  JPO  estimates.  The  2004  House 
Appropriations  Committee  defense  budget  report  claimed  that  the  lower  number  was  only  a 
“50th  percentile”  estimate  conducted  prior  to  the  concept  definition  phase,  and  thus  should 
be  treated  as  optimistic.  The  committee  report  also  alleged  that  the  CAIG  estimate  did  not 
take  into  account  a  full  “objective  SBR  constellation  of  21-24  satellites,”  the  total  costs  of 
which  “ could  exceed  $60  billion  based  on  the  current  understanding  of  program  requirements 
and  technology”  (U.S.  House  of  Representatives,  2004,  p.  313,  italics  in  original).  The 
authors  of  this  RAND  monograph  have  not  attempted  to  assess  independently  the  accuracy 
of  any  of  these  estimates,  but  merely  report  the  range  of  estimates  that  was  being  debated 
publicly  at  the  time.  See  also  “Appropriators  Slash  Space-Based  Radar  Funding”  (2004). 

17  SBIRS  High,  which  originally  got  under  way  in  the  mid-1990s  with  very  high  expecta¬ 
tions  as  an  acquisition  lead  pilot  program,  ended  up  experiencing  significant  cost  growth  and 
schedule  slippage.  Just  as  SBR  was  standing  up,  the  Air  Force  announced  the  need  to  pour 
additional  money  into  SBIRS,  and  even  examined  the  possibility  of  procuring  alternative 
systems.  Congress  reacted  adversely  toward  space  programs  in  general  when  confronted  with 
the  need  to  authorize  even  more  money  for  SBIRS  (Butler,  2004a).  For  a  brief  review  of  the 
early  history  of  SBIRS  as  an  acquisition  reform  program,  see  Lorell  and  Graser  (2001). 
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truth  in  costing  effort  to  reduce  the  chances  of  contractor  buy-in  or 
“low-ball”  estimates.  More  on  this  approach  is  described  below. 

Given  the  range  of  challenges  discussed  previously  confronting 
the  program,  particularly  the  difficulties  of  achieving  consensus  over 
the  overall  objective  system  requirements  and  the  specific  technolo¬ 
gies  to  employ,  it  appeared  to  program  managers  early  on  that  an  EA 
strategy  using  a  spiral  development  process  would  be  a  possible  solu¬ 
tion  to  many  of  the  challenges  confronted  by  the  program.  On  the 
face  of  it,  the  SBR  program  was  tailor-made  for  use  of  a  spiral  devel¬ 
opment  implementation  strategy,  since  such  a  strategy  is  designed  for 
managing  a  program  with  uncertain  end  requirements  and  uncertain 
projections  of  future  technology  maturity  in  key  areas.  In  addition,  a 
central  acquisition  goal  of  the  SBR  program  had  always  been  the 
rapid  fielding  of  an  operationally  useful  threshold  system  that  was  as 
affordable  as  possible — another  major  objective  of  EA. 

Thus,  to  achieve  its  goals  in  the  challenging  circumstances  con¬ 
fronting  SBR,  program  managers  adopted  an  evolutionary  approach 
to  acquisition  of  the  SBR  system,  which  involved  fielding  an  opera¬ 
tionally  useful  Increment  1  system  as  soon  as  possible  while  deferring 
all  nonessential  technologies  and  capabilities  to  later  increments.  As 
well  as  speeding  up  the  fielding  of  the  system,  program  managers  be¬ 
lieved  this  approach  allowed  them  to  minimize  life-cycle  costs  by 
waiting  until  certain  technologies  were  more  mature,  and  therefore 
less  risky  and  less  likely  to  promote  cost  growth,  before  inserting 
them.  Thus  a  key  element  of  their  strategy  was  to  invest  in  a  robust 
system  infrastructure  that  would  be  able  to  support  new  capability¬ 
enhancing  technologies  if  and  when  they  were  developed.  The  pro¬ 
gram  office  reasoned  that  extra  up-front  investments  in  technology 
maturation,  as  well  as  in  nonrecurring  hardware  development  to 
make  it  more  upgradeable  later  as  the  technologies  matured,  could 
pay  off  handsomely  in  overall  reduced  program  LCC. 

Unfortunately,  the  JPO’s  experience  to  date  with  this  approach 
has  been  decidedly  mixed.  This  is  primarily  because  the  current 
regulatory,  statutory,  and  political  environments  make  implementa¬ 
tion  of  a  true  spiral  approach  on  a  high-profile  program  such  as  SBR 
extremely  difficult.  The  pressures  placed  on  cost  analysts  and  pro- 
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gram  managers  to  generate  detailed  technical  descriptions  and  cost 
estimates  for  the  Increment  1  objective  system  made  it  nearly  impos¬ 
sible  to  apply  the  spiral  development  process  to  the  initial  increment. 

SBR  Increment  1:  Difficult  Choices.  As  noted  previously,  SBR  is 
potentially  a  very  large  and  costly  program,  which  led  to  considerable 
controversy.  In  2004,  the  DoD  CAIG  estimated  that,  if  all  costs  asso¬ 
ciated  with  Increment  1  were  included,  the  total  could  well  reach  $35 
billion  through  2025.  According  to  the  JPO,  one  reason  that  both  the 
CAIG  and  the  SBR  JPO  estimates  were  so  large  is  because  the  JPO, 
in  response  to  the  Young  Panel  recommendations  and  NSSAP  03-01, 
adopted  a  rigorous  truth  in  costing  approach  that  includes  a  wide  va¬ 
riety  of  cost  factors  routinely  excluded  in  the  cost  estimates  for  other 
programs.  For  example,  the  JPO  estimates  include  costs  associated 
with  integration,  assembly,  and  testing  for  all  of  the  105  different 
nodes  to  which  SBR  was  expected  to  direct  information,  as  well  as 
costs  associated  with  a  reference  architecture  (for  initial  costing  pur¬ 
poses)  that  consisted  of  a  notional  9  to  10  satellite  medium  earth  or¬ 
bit  (MEO)  constellation  plus  13  replenishment  SVs.18  The  cost  esti¬ 
mates  also  include  many  other  categories  of  costs  that  were  not 
typically  included  in  any  space  acquisition  programs,  such  as  costs 
associated  with  housing  of  ground  systems  personnel.19 

The  JPO’s  original  concept  of  Increment  1  followed  the  spiral 
development  approach  based  on  extensive  trade  studies,  experimenta¬ 
tion,  and  feedback  loops  that  would  lead  to  an  evolution  of  the  final 
Increment  1  concept  and  objective  end  requirements  as  the  program 
progressed.  Using  a  notional  Increment  1  plan  of  9  to  10  SVs  plus  13 
replenishment  SVs  as  a  planning  example,  the  JPO  concept  proposed 
a  strategy  of  upgrading  the  SVs  in  blocks  throughout  Increment  1  as 
the  technologies  matured  and  the  requirements  were  refined.  Thus  a 


18  In  July  2005,  the  government  adopted  an  official  baseline  for  the  program  called  the 
Space  Radar  Government  Reference  Architecture  (GRA).  The  GRA  consists  of  a  nine- 
satellite,  low,  earth-orbit  constellation. 

19  As  noted  previously,  critics  in  Congress  and  elsewhere  were  skeptical  of  the  cost  estimates 
produced  by  the  JPO  and  the  CAIG  during  this  time  frame  and  criticized  some  of  the  no¬ 
tions  underlying  the  baseline  assumptions  from  which  the  cost  estimates  were  derived. 
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stair-step  spiral  approach  on  hardware  and  software  was  advocated  in 
order  to  build  up  to  a  notional  IOC  for  Increment  1  of  roughly  four 
SVs.  Future  improvements  during  Increment  1  would  build  on  this 
basic  capability  with  upgraded  hardware,  eventually  producing  a  no¬ 
tional  constellation  of  8  to  14  SVs  to  meet  the  threshold  Increment  1 
requirement.  As  noted  previously,  an  official  GRA  baseline  was  es¬ 
tablished  in  July  2005  that  consisted  of  a  nine-satellite,  low,  earth- 
orbit  constellation. 

The  JPO  cautioned  that  much  more  money  would  have  to  be 
invested  up  front  than  in  the  past  to  define  end  capabilities,  flesh  out 
notional  spirals,  conduct  ongoing  technology  maturity  assessments, 
and  design  and  engineer  the  initial  SVs  to  facilitate  later  technology 
insertion.  JPO  cost  estimates  included  six  to  seven  percent  more  in 
additional  nonrecurring  engineering  costs  to  develop  more  robust 
buses  and  SVs  to  facilitate  the  later  incorporation  of  block  upgrades. 
Some  nonrecurring  funding  was  also  included  for  beginning  devel¬ 
opment  of  Increment  2  capabilities  during  Increment  1.  The  JPO 
estimated  that  this  additional  up-front  expense  could  save  up  to  30 
percent  in  nonrecurring  engineering  costs  later  in  the  program  when 
technology  matured  and  technology  insertions  and  block  upgrades 
were  made  later  in  Increment  1 .  This  approach  was  combined  with 
an  incremental  capability  growth  plan  based  on  what  the  JPO  called 
preplanned  decision  opportunity  technology  on-  and  off-ramps, 
where  decisions  about  major  system  upgrades  and  technology  inser¬ 
tions  could  be  made  based  on  technology  maturity,  experience  with 
the  existing  SVs  through  feedback  loops,  and  other  factors. 

For  developing  the  actual  as  opposed  to  notional  content  of  In¬ 
crement  1 ,  the  original  SBR  Phase  A  acquisition  strategy  envisioned 
competing  two  contractors  to  refine  multiple  alternative  Increment  A 
concepts  during  2004.  Each  contractor  would  then  mature  and  select 
a  single  preferred  Increment  1  concept  in  support  of  a  formal  system 
requirements  review  (SRR)  and  system  design  review  (SDR)  in  early 
2005,  with  a  final  down  select  to  one  contractor  in  2006.  In  support 
of  this  approach,  the  JPO  awarded  both  Lockheed  Martin  and 
Northrop  Grumman  two-year,  $220  million  concept  development 
contracts  in  April  2004. 
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The  JPO  was  not  granted  the  resources  to  implement  this  strat¬ 
egy,  however.  Shortly  after  the  contract  award,  the  House  Appropria¬ 
tions  Committee  gutted  SBR  by  cutting  over  75  percent  of  the  ad¬ 
ministration’s  requested  funding  for  FY  2005. 20  The  committee’s 
report  claimed  the  program  was  too  risky  technologically,  too  costly, 
and  not  likely  to  achieve  the  needed  capabilities  at  the  projected  cost. 
It  concluded  that  “[wjithout  a  new  approach,  the  committee  sees  lit¬ 
tle  future  for  the  Space  Based  Radar.”  Shortly  thereafter,  the  Senate 
Appropriations  Committee  also  recommended  a  major  cut  in  the 
program.21  A1  though  DoD  and  the  Air  Force  fought  to  restore  fund¬ 
ing,  the  final  conference  report  filed  at  the  end  of  July  retained  the 
large  cuts  recommended  by  the  House  Appropriations  Committee, 
dealing  a  blow  to  SBR  characterized  by  the  defense  press  as  “crip¬ 
pling”  or  “fatal”  (Butler,  2004b).  The  conference  report  ordered  the 
Air  Force  to  end  SBR  as  a  formal  acquisition  program,  and  use  the 
remaining  $75  million  in  funding  for  technology  demonstration,  risk 
reduction,  and  concept  development  (Butler,  2004c). 

In  early  2005,  press  accounts  reported  that  DoD  and  the  Air 
Force  would  propose  a  revised  SBR  effort  that  would  restructure  the 
first  phase  of  the  program  into  what  amounted  essentially  to  an  ex¬ 
tended  technology  demonstration  and  concept  development  effort. 
Air  Force  Under  Secretary  Peter  Teets  noted  that  Congress  had  to  be 
presented  with  “a  clearer,  more  affordable  path  forward”  for  SBR 
(Rees,  2005).  Central  to  the  effort  would  be  the  development  of  a 
more  detailed  concept  of  operations  (CONOPS),  which  then  could 
lead  to  the  launch  of  a  single  demonstration  satellite  in  the  2008- 
2010  time  frame.  Concept  and  CONOPS  development  would  focus 
on  more  effectively  defining  and  integrating  the  requirements  of  the 
warfighter  and  intelligence  communities,  the  two  major  system  users. 
Following  the  conclusion  of  a  successful  multiyear  demonstration 


20  Nearly  $253  million  were  cut  from  the  requested  $327.7  million  (“Appropriators  Slash 
Space-Based  Radar  Funding,  Criticize  Costs,”  2004). 

21  The  Senate  version  cut  $100  million  from  the  requested  $327.7  million  (Tuttle,  2004). 
The  final  program  cut  settled  on  by  Congress  was  $252.7  million,  leaving  only  $73.8  million 
remaining  of  the  original  FY  2005  request  (U.S.  Department  of  the  Air  Force,  2005,  p.  749). 
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program  with  a  single  satellite,  a  new  SBR  program  could  be 
launched  that  is  aimed  at  the  acquisition  of  a  full  constellation  of  op¬ 
erational  SVs. 

Formal  submission  of  the  restructured  program  to  Congress 
took  place  in  February  2005.  To  underline  the  extent  of  the  program 
restructuring,  the  program  was  renamed  the  Space  Radar  (SR).  The 
SR  program  is  essentially  a  concept  development  and  technology 
demonstration  effort,  but  remains  a  large-scale  program.  The  central 
goal  of  the  restructured  SR  program  was  to  increase  the  near-term 
program  focus  on  technology  risk  reduction,  more  precise  require¬ 
ments  definition,  and  technology  demonstration.  Special  areas  of  em¬ 
phasis  for  risk  reduction  efforts  included  the  electronically  scanned 
array  (ESA)  and  on-board  processing.  Technology  risk  reduction  ef¬ 
forts  were  envisioned  to  climax  with  on-orbit  demonstrations.  De¬ 
spite  the  dramatic  congressional  funding  cuts  mandated  by  Congress 
for  FY  2005,  the  restructured  SR  effort  proposed  by  the  Air  Force 
envisions  a  rapid  spool-up  of  funding  in  the  out  years.  Air  Force 
planners  projected  spending  to  rise  to  over  $1  billion  in  FY  2009,  and 
to  increase  to  over  $1.4  billion  in  FY  2011.  Air  Force  budget  docu¬ 
ments  projected  total  spending  on  the  program  (SBR  plus  SR)  of 
nearly  $5.2  billion  through  FY  2011  (U.S.  Department  of  the  Air 
Force,  2005,  pp.  749,  751-752). 

SBR  and  EA.  The  SBR  JPO  originally  adopted  an  EA  approach 
to  help  manage  a  very  large  and  complex  program  with  multiple 
stakeholders,  where  achievement  of  consensus  on  threshold  and  ob¬ 
jective  requirements,  CONOPS,  system  architecture,  technology  ma¬ 
turity  and  risk,  technology  insertion  and  phasing,  and  a  variety  of 
other  key  program  variables  was  difficult  to  achieve.  In  principle,  an 
EA  strategy  employing  spiral  development  appears  to  be  a  sensible 
strategy  to  adopt  in  such  challenging  circumstances.  Time  and  ex¬ 
perimentation  were  clearly  needed  to  evolve  the  program  forward  and 
clarify  a  variety  of  key  issues.  So  what  went  wrong  with  the  original 
SBR  effort? 

According  to  program  managers,  the  existing  statutory,  regula¬ 
tory,  and  budgetary  environments  make  implementation  of  a  spiral 
development  approach  on  a  large,  high-profile  program  such  as  SBR 
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extremely  challenging.  Particularly  when  a  program  is  under  close 
scrutiny  from  Congress,  budget  planners  and  cost  analysts  tend  to 
press  for  a  level  of  certainty  and  detail  for  the  entire  Increment  1  pro¬ 
gram  that  violates  the  Spiral  Development  concept.  While  admitting 
that  the  program  may  have  been  “overly  ambitious”  in  its  early 
claims,  and  might  have  been  more  clearly  defined  at  the  beginning, 
program  managers  complained  that  the  budget  planners  and  cost 
analysts  insisted  on  having  precise  definitions  of  what  the  system 
hardware  and  software  would  look  like  ten  years  out.  In  a  like  man¬ 
ner,  requirements  managers  tended  to  proliferate  key  design  parame¬ 
ters  very  early  in  the  program.  The  result,  in  the  view  of  some  JPO 
managers,  was  that  all  the  program  flexibility  and  positive  uncertain¬ 
ties  that  are  inherent  in  a  spiral  development  approach  were  con¬ 
stantly  under  attack  from  the  earliest  phases  of  the  program. 

The  EA  process,  particularly  using  spiral  development,  presumes 
a  high  degree  of  flexibility  in  budgeting,  cost,  and  requirements,  par¬ 
ticularly  early  in  the  program.  According  to  JPO  managers,  the  cur¬ 
rent  acquisition  process  does  not  permit  such  an  approach.  Indeed, 
SBR  rapidly  evolved  from  a  spiral  development  program  to  an  incre¬ 
mental  development  program,  then  to  something  approaching  a  tra¬ 
ditionally  structured  single-step-to-capability  program  with  internal 
steps  or  block  upgrades.  Once  it  became  more  like  a  traditionally 
structured  program,  its  uncertainties  became  major  deficits  that 
opened  it  to  criticism  from  many  quarters. 

It  may  be  that  a  spiral  development  approach  was  not  feasible  on 
any  potentially  large-scale,  high-profile  program  such  as  SBR,  par¬ 
ticularly  in  a  difficult  political  environment  such  as  existed  at  the  be¬ 
ginning  of  SBR,  which  was  dominated  by  the  specter  of  large  cost 
overruns  and  schedule  slippage  with  major  space  programs  such  as 
SBIRS.  As  General  Lance  Lord,  Commander  of  Air  Lorce  Space 
Command,  noted,  “We  set  the  [SBR  program]  structure  up  so  that 
we  could  do  technology  and  risk  reduction  and  have  a  good  contrac¬ 
tor  team  together  with  the  CONOPS  [development].”  The  problem, 
according  to  General  Lord,  was  that  “some  people  were  taking  what’s 
happened  in  the  SBIRS  program  and  saying,  ‘Okay,  if  that’s  the  cost 
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growth  you  got  in  that,  then  you  know  what  [is  going  on]  in  other 
programs’”  (Tuttle,  2004,  p.  5). 

At  the  time  of  the  original  congressional  cuts,  some  senior  Air 
Force  officials  began  publicly  questioning  whether  the  spiral  devel¬ 
opment  approach  should  be  used  on  any  large-scale  hardware  devel¬ 
opment  programs.  For  example,  in  July  2004,  Air  Force  Major  Gen¬ 
eral  William  Shelton,  U.S.  Strategic  Command’s  policy,  resources, 
and  requirements  director,  publicly  cautioned  against  the  overuse  and 
misapplication  of  spiral  development  in  space  programs.  General 
Shelton  argued  that  spiral  development  was  applicable  to  software 
development,  but  cautioned  against  its  use  on  expensive,  relatively 
small,  specialized  space  constellations  such  as  SBR:  “You  really  have 
to  think  through  spiral  development  for  smaller  constellations”  (“Spi¬ 
ral  Out  of  Control?”  2004).  Under  Secretary  Teets  probably  put  his 
finger  on  the  key  problem  with  the  use  of  spiral  development  on  pro¬ 
grams  such  as  SBR  when  he  noted  in  early  2005,  “I  think  one  of  the 
things  last  year  that  kind  of  got  us  in  a  little  trouble  in  terms  of  get¬ 
ting  the  program  properly  funded  from  Congress  is  that  we  didn’t 
define  clearly  the  size  and  scope  of  the  Space-Based  Radar  program 
that  we  were  proposing”  (Rees,  2005).  Thus,  the  very  flexibility  pro¬ 
gram  managers  were  seeking  to  provide  the  tools  to  resolve  uncertain¬ 
ties  opened  up  the  program  to  criticism  and  skepticism.  In  short, 
what  may  appear  to  be  prudent  flexibility  and  sensible  openness  to 
evolutionary  change  to  program  managers  may  appear  as  open-ended 
and  undisciplined  buy-ins  by  some  members  of  Congress  and  other 
observers.22 

Is  evolutionary  acquisition,  then,  not  usable  on  such  programs  as 
SBR?  The  answer  is  probably  not,  if  the  program  is  heavily  dependent 
on  relatively  new  and  immature  technologies,  and  continuing  debate 
exists  over  system  concepts,  CONOPS,  architectures,  requirements, 


22  Of  course,  another  major  area  of  concern  of  critics  of  SBR  was  the  actual  level  of  maturity 
of  the  technologies  that  were  planned  for  near-term  insertion  into  the  system.  In  July  2004, 
the  GAO  issued  a  report  to  Congress  on  SBR  that  strongly  recommended  that  DoD  delay 
the  product  development  stage  of  SBR  to  permit  fuller  assessment,  risk  reduction,  and  matu¬ 
ration  of  the  key  technologies  involved  in  SBR.  This,  of  course,  is  precisely  what  eventually 
happened  (U.S.  Government  Accountability  Office,  2004). 
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capabilities,  technologies,  and  cost  effectiveness.  Remember,  a  fun¬ 
damental  component  of  EA,  at  least  in  the  minds  of  many  early  advo¬ 
cates,  was  that  each  spiral  or  increment  should  focus  on  using  rea¬ 
sonably  mature  technologies  to  take  small  steps  toward  a  modest  new 
level  of  operationally  useful  capabilities.  In  the  minds  of  some  of  its 
opponents,  SBR  was  planning  on  drawing  on  too  many  high-risk, 
lower-maturity  technologies,  even  in  the  early  stages  of  its  first  incre¬ 
ment.  In  their  view,  SBR  Increment  1  had  already  evolved  into  a 
huge  and  technologically  risky  advance  in  capabilities,  thus  violating 
the  EA  concept  of  small  incremental  improvements. 

EA  using  spiral  development  may  be  more  feasible,  however,  in 
a  variety  of  other  circumstances.  One,  of  course,  may  be  in  technol¬ 
ogy  demonstration  programs.  Clearly  the  Air  Force  has  adopted  a 
posture  of  restructuring  the  initial  phase  of  a  new  SR  effort  as  a  tech¬ 
nology  demonstrator  effort.  If  this  leads  to  the  successful  launch  of  a 
new  acquisition  program,  based  on  more  mature  technologies  and 
more  precisely  defined  objective  system  capabilities,  then  one  could 
argue  that  this  first  phase  is  a  form  of  spiral  development,  and  that 
the  process  works. 

Other  areas  where  this  approach  may  still  be  used  successfully 
on  large,  high-visibility  programs  are  in  environments  outside  the 
normal  DoD  acquisition  regulatory  and  statutory  environment.  The 
KEI  effort  may  be  one  such  program. 

KEI:  Capability-Based  EA  in  an  Acquisition  Reform  Environment 

Introduction.  The  KEI  (Kinetic  Energy  Interceptor)  missile  de¬ 
fense  program  was  initiated  in  2002  by  the  Missile  Defense  Agency 
(MDA).  The  goal  of  the  program  is  to  design,  develop,  and  deploy 
mobile,  kinetic  energy-based  missiles  that  can  intercept  and  destroy 
enemy  ballistic  missiles.  Initially,  the  KEI  was  intended  to  intercept 
missiles  during  their  boost  phase,  which  can  last  as  little  as  180  sec¬ 
onds  (Barnard,  2004).  Now,  however,  it  is  seen  as  the  basis  for  a  mul¬ 
tipurpose  interceptor  that  can  also  destroy  missiles  in  their  ascent 
phase  (500-600  seconds  after  launch)  and  midcourse  phase  (includ¬ 
ing  the  apex  of  flight  at  about  1,200  seconds  after  launch).  Current 
efforts  focus  on  a  ground-based  system,  but  eventually  the  KEI  is  ex- 
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pected  to  perform  boost-phase  intercepts  from  sea-based  and  perhaps 
space-based  launch  platforms.  According  to  a  draft  environmental 
assessment  issued  in  September  2004,  MDA  “contemplates  the  de¬ 
velopment  of  a  space-based  test  bed”  in  2012  (“Missile  Defense 
Agency  Eyeing  Space-Based  Interceptor  Test  Bed  by  2012,”  2004). 

KEI  differs  from  the  programs  discussed  previously  in  this  chap¬ 
ter  in  at  least  three  important  respects.  First,  and  perhaps  most  im¬ 
portant,  KEI  is  being  developed  by  MDA,  which  is  a  separate  DoD 
agency  that  operates  in  accordance  with  its  own  unique  acquisition 
environment.  In  May  1993,  the  Strategic  Defense  Initiative  Organi¬ 
zation  (SDIO),  founded  during  the  Reagan  administration,  was  recast 
as  the  Ballistic  Missile  Defense  Organization  (BMDO),  and  was  re¬ 
structured  to  report  directly  to  the  Under  Secretary  of  Defense  for 
Acquisition.  Almost  a  decade  later,  the  organization’s  unique  position 
was  further  enhanced  when  Secretary  of  Defense  Donald  Rumsfeld 
upgraded  the  status  of  the  organization  to  that  of  a  DoD  agency  and 
renamed  it  the  Missile  Defense  Agency  (MDA) .  MDA  is  chartered  by 
the  President  and  mandated  by  Congress.  MDA  acquisition  programs 
are  subject  to  the  normal  DoD  5000  regulatory  guidance,  rather  than 
NSSAP  03-01.  However,  as  a  DoD  agency  with  a  unique  status, 
MDA  program  managers  can  often  waive  acquisition  directives  issued 
as  part  of  the  DoD  5000  series.  This  often  permits  far  more  flexibility 
in  the  formulation  and  execution  of  acquisition  strategies  than  is 
typical  within  the  normal  service  acquisition  organizations. 

Second,  by  definition  and  by  charter,  all  MDA  acquisition  pro¬ 
grams  are  aimed  at  contributing  to  the  achievement  of  a  single,  well- 
defined,  relatively  narrow  mission  objective:  “to  develop,  test  and 
prepare  for  deployment  of  a  missile  defense  system”  (Missile  Defense 
Agency,  undated).  The  fundamental  underlying  technology  has  al¬ 
ready  been  determined.  MDA  has  decided  that  component  missile 
defense  systems  will  be  “primarily  based  on  hit-to-kill  technology” 
(Missile  Defense  Agency,  undated).  In  addition,  MDA  operates  with 
a  single  chain  of  command,  with  clear  lines  of  authority.  The  MDA 
Director  is  the  acquisition  executive  for  all  Ballistic  Missile  Defense 
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systems  and  programs.23  All  specific  systems  under  development  are 
meant  to  fit  into  a  larger  system  of  systems  that  contributes  to  the 
single  overarching  mission  of  MDA.  Specific  component  systems  and 
funding  can  be  traded  off  against  one  another  by  the  director  and 
senior  MDA  management  to  best  achieve  the  overall  MDA  mission. 
Thus,  in  contrast  to  SBR  and  some  other  space  programs,  MDA  ac¬ 
quisition  programs  are  executed  in  a  highly  flexible  procurement  en¬ 
vironment,  with  a  reasonably  clearly  defined  consensus  mission  re¬ 
quirement,  agreement  on  the  basic  technological  approach,  and  under 
a  single  chain  of  command  with  clear  lines  of  authority. 

Finally,  the  director  of  the  KEI  program,  Terry  Little,  has  long 
been  a  strong  advocate  for  acquisition  reform,  and  the  structure  of 
KEI  reflects  several  different  acquisition  reform  ideas  he  has  helped  to 
pioneer.  The  more  flexible  MDA  acquisition  environment  permits 
greater  latitude  for  innovative  program  managers  such  as  Terry  Little 
to  experiment  with  novel  acquisition  approaches,  including  concepts 
such  as  EA.  Interestingly,  however,  even  Terry  Little  has  backed  off 
on  some  of  the  original  elements  of  spiral  development  as  originally 
conceived  in  implementing  the  KEI  effort. 

Nonetheless,  KEI  is  clearly  structured  as  an  EA  effort,  with  the 
central  focus,  like  other  EA  programs,  on  rapid  deployment  of  initial, 
operationally  useful  capabilities.  An  EA  strategy  is  used  to  mitigate 
schedule,  technological,  and  cost  risk  by  producing  more  modest  ca¬ 
pability  blocks  in  increments,  and  by  focusing,  in  principle,  on  more 
mature  technologies.  According  to  senior  program  managers,  the  KEI 
program  structure  has  been  derived  by  asking  the  following  question: 
What  meaningful  part  of  a  threat  can  be  addressed  with  existing  in¬ 
dustry  capabilities  and  within  the  required  schedule? 

Thus,  KEI  might  be  characterized  as  emphasizing  schedule  as  an 
independent  variable  (SAIV)  as  well  as  CAIV.24  Thus  increment 
threshold  system  requirements  are  driven  by  schedule  realism  (what 


23  As  of  early  2005,  the  MDA  Director  was  Air  Force  Lt  Gen  Henry  A.  “Trey”  Obering  III. 

24  SAIV  is  a  play  on  words  (or  acronyms)  used  by  the  authors  in  reference  to  the  well-known 
acquisition  reform  concept  of  the  1990s  called  CAIV,  or  cost  as  an  independent  variable. 
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operationally  useful  capability  can  be  realistically  achieved  within  a 
reasonably  short  schedule?)  and  by  cost  realism  (what  operationally 
useful  capability  can  be  realistically  achieved  within  the  budgeted  re¬ 
sources?). 

In  addition,  program  managers  have  established  some  funda¬ 
mental  rules,  based  on  past  acquisition  reform  experience,  designed  to 
help  them  achieve  the  hoped  for  EA  benefits  while  avoiding  the  pit- 
falls  of  requirements  creep,  and  high  life-cycle  costs: 

•  Avoid  user  pressure  to  meet  the  entire  threat  in  one  step. 

•  Treat  contractor  promises  to  meet  the  entire  threat  with  skepti¬ 
cism. 

•  Emphasize  manufacturability  and  software  readiness. 

Finally,  KEI’s  focus  on  evolutionary  requirements  and  capabili¬ 
ties  is  combined  with  the  radical  acquisition  reform  concept  of  price- 
based  acquisition  (PBA)  to  maintain  contractor  focus  on  cost  realism 
and  continuous  cost  reduction  (Lorell,  Graser,  and  Cook,  2005). 

KEI  Spirals:  A  Radically  Different  Approach.  The  KEI  program  is 
notionally  divided  into  four  spiral  elements  that  correspond  to  differ¬ 
ent  launch  platforms  and  different  intercept  phases  of  the  target  mis¬ 
sile  trajectory: 

•  Spiral  1:  Ground-based  mobile  boost/ascent-phase  interceptor 

•  Spiral  2:  Transition  to  sea-based  interceptor 

•  Spiral  3:  Ground/sea-based  midcourse  interceptor 

•  Spiral  4:  Space-based  interceptor. 

The  KEI  spiral  1  system,  which  is  the  only  system  that  had  been 
considered  in  any  detail  as  of  mid-2004,  consists  of  three  compo¬ 
nents:  a  command,  control,  battle  management,  and  communications 
(C2BMC)  system,  a  launcher,  and  a  missile.  In  December  2003, 
MDA  awarded  Northrop  Grumman  and  Raytheon  an  eight-year, 
$4.6  billion  development  contract.  The  program  office  plans  to 
award  the  winning  contractor  a  firm-fixed-price/incentive-fee  con¬ 
tract,  with  a  warranty,  for  each  of  the  three  system  components  for 
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Spiral  1.  The  program  office  aims  at  fielding  the  ground-based  mo¬ 
bile  interceptor  in  the  2012-2013  time  frame. 

In  a  radical  departure  from  normal  DoD  practice,  KEI’s  PBA 
approach  means  that  the  unit  price  for  each  component  will  be  inde¬ 
pendent  of  the  quantity  produced.  As  part  of  the  down  select,  the 
competing  contractors  were  required  to  offer  an  option  for  a  single 
firm  fixed  unit  price  for  all  units  during  development,  no  matter  the 
number  that  are  eventually  procured.  The  approach  of  combining 
competition  plus  the  requirement  for  a  single  firm  fixed  price  with  a 
warranty  emulates  commercial  market  conditions  that  encourage  con¬ 
tractors  to  focus  on  price,  quality,  and  manufacturability.  The  KEI 
SPO  evaluated  the  initial  cost  and  schedule  estimates  submitted  by 
the  competing  contractors  (although  a  Truth  in  Negotiations  Act 
[TINA]  waiver  had  been  received),  and  adjusted  them  in  order  to 
achieve  greater  schedule  realism. 

Like  the  other  programs  examined  in  this  chapter,  KEI  is  still  at 
a  very  early  stage  of  development  and  design.  The  spiral  program 
structure  is  still  notional.  And  in  spite  of  its  innovative  approach  and 
its  flexible  acquisition  environment,  KEI  has  not  been  free  of  contro¬ 
versy.  Indeed,  in  July  2004,  at  the  same  time  Congress  slashed  fund¬ 
ing  for  SBR,  the  Senate  proposed  cutting  nearly  50  percent  of  the 
KEI  FY  2006  budget  request.  While  the  KEI  acquisition  program 
generally  escaped  major  public  criticism  comparable  to  SBR,  many 
senators  expressed  deep  concerns  about  the  proposed  basing  mode  for 
the  ground-based  KEI.25  The  proposed  cuts  were  sustained  in  the 
conference  report  (Ruppe,  2004). 

Although  KEI  has  hardly  been  free  of  controversy  and  has  suf¬ 
fered  significant  funding  cuts  (with  more  possibly  coming  in  the  fu¬ 
ture),  little  of  the  criticism  has  been  directed  against  its  innovative 
acquisition  strategy  and  its  planned  use  and  implementation  of  EA. 
Unlike  SBR,  which  Congress  explicitly  directed  to  be  cancelled  as  a 


25  The  criticism  is  based  on  the  concern  that  the  KEI  ground-based  interceptor  allegedly 
must  be  located  unreasonably  close  to  the  launch  points  in  the  countries  of  several  potential 
adversaries  in  order  to  intercept  effectively  during  the  boost  phase  (see,  for  example,  Selinger, 
2004). 
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formal  acquisition  program,  KEI  is  expected  to  be  delayed  only  by 
funding  cuts,  while  continuing  forward  as  a  formal  acquisition  ef¬ 
fort.26  Therefore,  whatever  the  merits  or  lack  thereof  of  the  program 
with  respect  to  a  ground-based  boost-phase  interceptor,  and  whatever 
its  fate,  the  authors  believe  useful  lessons  can  still  be  garnered  from 
the  program’s  overall  history  through  2004  with  respect  to  imple¬ 
menting  EA.  To  a  lesser  or  greater  extent,  the  same  is  true  of  the 
other  four  case  studies  examined  here. 


Some  Lessons  Learned  from  Space  Case  Studies  for  EA 
Implementation 

The  Need  for  Additional  Up-Front  Spending 

All  the  program  officials  interviewed,  particularly  those  involved  in 
SBR  and  KEI,  emphasized  the  importance  of  planning  for  and  ob¬ 
taining  extra  up-front  spending  in  large-scale,  complex  EA  programs. 
This  requirement  arises  from  the  necessity  to  map  out  the  complex 
program  structure  implied  by  EA,  which  includes  a  series  of  separate 
overlapping  increments,  each  requiring  the  definition  of  operationally 
useful  threshold  and  objective  requirements;  and  the  additional  sys¬ 
tem  engineering  and  nonrecurring  engineering  to  support  the  pro¬ 
gression  of  upgrades  and  technology  insertion  that  takes  place  within 
and  between  increments.  Also,  additional  up-front  resources  are  re¬ 
quired  to  support  more  extensive  and  continuously  revised  evolution¬ 
ary  costing  efforts. 

Shifting  User  Feedback  Loops  to  the  OT&E  Phase 

All  program  managers,  particularly  those  in  the  SBR  and  KEI  pro¬ 
grams,  stressed  the  importance  and  difficulty  of  controlling  require- 


26  Some  observers  anticipate  further  funding  cuts  to  MDA  in  2005.  There  continue  to  be 
advocates  for  cancellation  of  KEI  outside  ofDoD,  and  of  course  the  program  could  be  can¬ 
celled  in  the  future.  For  a  typical  argument  advocating  cancellation,  see  Dinerman  (2005). 
According  to  press  accounts,  draft  budget  documents  show  an  MDA  KEI  funding  request 
for  FY  2005  of  $230  million.  This  compares  to  an  anticipated  $1.1  billion  FY  2006  request 
for  KEI  projected  by  MDA  in  its  2004  budget  documents  (Capaccio,  2005). 
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ments  creep  and  add-on,  particularly  in  an  evolutionary  acquisition 
environment.  Initial  conceptual  versions  of  EA  strategies  using  spiral 
development  envisioned  constantly  functioning  feedback  loops  from 
the  user  community  to  fine-tune  requirements  and  make  sure  that 
developers  produced  end  products  that  meet  real  needs  in  the  field. 
But  actual  experience  in  the  early  phases  of  the  case  study  programs 
described  here  suggest  that  uncontrolled  feedback  in  the  early  concept 
development  stages,  particularly  on  programs  with  multiple  user 
communities  such  as  SBR,  can  lead  to  a  counterproductive  piling  up 
of  sometimes  mutually  inconsistent  requirements,  concepts,  and 
technologies. 

The  KEI  program  managers  particularly  stressed  the  importance 
of  restricting  the  feedback  loop  process  to  the  OT&E  phase  when 
actual  hardware  exists  and  can  be  evaluated.  In  the  view  of  the  KEI 
program  managers,  constant  uncontrolled  early  feedback  from  the 
user  undermines  the  fundamental  benefits  of  EA,  because  the  user 
communities  tend  to  press  for  the  full  achievement  of  maximum  ob¬ 
jective  requirements  from  the  beginning,  thus  pushing  programs  to¬ 
ward  more  traditional  single-step-to-capability  program  structures. 
This  problem  is  exacerbated  when  multiple  user  communities  are  in¬ 
volved,  as  in  the  case  of  SBR. 

"Logistics  Takes  It  in  the  Shorts" 

In  the  colorful  phrase  of  one  program  manager,  it  is  crucial  for  EA 
program  managers  to  recognize  and  plan  for  the  fact  that  “logistics 
takes  it  in  the  shorts”  in  EA  programs.  What  is  meant  by  this  colorful 
phrase  is  that  EA  greatly  complicates  logistics  planning  (and  life-cycle 
cost  analysis)  by  leading  to  a  proliferation  of  different  system  configu¬ 
rations  as  the  system  evolves  through  its  increments  or  spirals.  The 
best  approach  to  dealing  with  this  challenge,  in  the  view  of  several 
program  managers,  is  to  plan  from  the  beginning  to  back-fit  earlier 
variants  to  bring  them  up  to  the  standard  of  the  latest  configurations, 
or  merely  to  replace  old  variants  with  the  current  versions.  The  KEI 
program  manager  suggests  a  “blast  down”  solution  for  old  variants, 
where  the  earlier  versions  are  consumed  through  use  as  test  vehicles 
for  later  stages  of  the  program.  Whatever  the  approach,  budget  plan- 
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ners,  cost  analysts,  and  logistics  planners  must  be  prepared  to  antici¬ 
pate  and  plan  for  the  additional  complexities  and  costs  that  will  be 
incurred  through  the  use  of  EA  by  the  fielding  of  multiple  versions 
and  configurations  of  the  same  system. 

Focus  on  Capability  Objectives,  Not  Traditional  Requirements 

In  the  view  of  several  program  managers,  EA  provides  a  structure  and 
a  process  to  achieve  the  long-sought  acquisition  reform  objective  of 
focusing  on  capability  objectives.  According  to  this  view,  programs 
should  seek  to  develop  and  acquire  a  well-defined,  needed,  and  realis¬ 
tic  mission  capability  rather  than  focusing  on  meeting  detailed  tech¬ 
nical  requirements  established  at  program  inception.  Such  an  ap¬ 
proach — which  gives  the  SPO  and  the  contractor  greater  flexibility  to 
seek  innovative  technical  and  operational  solutions  to  help  meet  well- 
defined  mission  and  capability  requirements — can,  in  principle,  be 
facilitated  by  EA. 

As  programs  move  toward  more  rigidly  defined  and  narrow 
technical  requirements  definitions  and  key  performance  parameters 
(KPPs),27  they  tend  to  migrate  toward  the  more  traditional  single- 
step-to-capability  approach.  In  the  view  of  the  KEI  program  manag¬ 
ers,  the  SPO  should  emphasize  mission  assurance  (as  stressed  in  the 
Young  Panel  findings)  and  schedule  realism  above  any  rigid  adher¬ 
ence  to  narrow  technical  requirements.  Unfortunately,  existing  regu¬ 
latory  and  political  pressures  tend  to  push  programs  strongly  in  the 
latter  direction,  according  to  the  SBR  program  managers  and  others. 

Ironically,  establishment  of  KPPs  was  a  crucial  element  of  some 
of  the  early  acquisition  reform  pilot  programs  in  the  early  1990s,  such 
as  JDAM  and  JASSM.  The  point  then  was  to  focus  the  program  of¬ 
fice  and  the  contractors  on  a  small  set  of  broad  but  critical  perform¬ 
ance  measures  that  would  orient  the  programs  toward  capability  ob- 


27  KPPs  were  introduced  during  the  burst  of  acquisition  reform  implemented  by  DoD  in  the 
late  1990s.  They  were  part  of  the  move  away  from  detailed  technical  military  specifications 
in  ORDs  toward  requirements  defined  in  performance  terms.  KPPs  are  nonnegotiable 
minimum  requirements,  which  cannot  be  traded  off  against  cost  or  schedule,  and  that  are 
usually  stated  in  performance  capability  terms  rather  than  in  technical  terms. 
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jectives  and  away  from  detailed  technical  requirements  and  specifica¬ 
tions  dictated  by  the  government.  Both  Terry  Little  at  the  KEI  pro¬ 
gram  office,  who  helped  develop  the  KPP  concept  on  the  JDAM  and 
JASSM  programs,  and  the  SBR  program  managers  are  vigorously  op¬ 
posed  to  the  KPP  concept  now.  This  is  because,  since  the  1990s,  the 
use  of  KPPs  has  drifted  away  from  the  original  focus  on  broad  mis¬ 
sion  capabilities  and  toward  the  old  concept  of  detailed  technical  per¬ 
formance  requirements.  Thus,  program  officials  at  SBR  and  elsewhere 
complained  that  highly  specific  and  narrow  KPPs  are  established  early 
in  the  program  and  then  rigidly  adhered  to,  thus  stifling  creative  solu¬ 
tions  that  emerge  later  and  limiting  the  ability  of  program  managers 
to  trade  off  small  amounts  of  capability  in  one  area  for  large  returns 
in  another  area,  or  in  cost  or  schedule  reduction. 

Evolutionary  Costing  Works  and  Is  Widely  Accepted  on  EA  Programs 

Nearly  all  the  cost  analysts  interviewed  who  were  involved  in  or  fa¬ 
miliar  with  these  case  studies  expressed  confidence  in  the  use  of  an 
evolutionary  costing  approach  on  EA  programs.  Program  managers 
noted  that  evolutionary  costing  is  the  only  feasible  and  realistic  ap¬ 
proach  to  use  on  EA  programs,  especially  those  that  employ  the  spiral 
development  process.  Evolutionary  costing  requires  that  the  cost 
analysts  work  closely  with  the  contractors  and  the  government  tech¬ 
nical  team  to  track  the  design  and  the  technologies  as  they  evolve. 
One  approach  that  is  popular  is  to  develop  cooperatively  a  joint  cost 
model  with  the  contractor,  based  on  mutually  agreed  upon  method¬ 
ologies,  baseline  assumptions,  and  cost  factors,  that  facilitates  the 
constant  updating  of  the  cost  estimates  as  the  design  evolves  and  new 
information  and  data  are  input  into  the  cost  model.28  The  inputs  to 
the  government  model  may  not  be  the  same  as  the  contractor’s 
model;  however,  joint  models  tend  to  make  the  discussion  and  recon¬ 
ciliation  of  the  cost  estimates  much  more  effective. 

In  virtually  every  case  examined,  the  cost  analysts  overwhelm- 
ingly  focused  their  efforts  on  the  initial  increment  or  spiral,  since 


28  This  approach  has  been  successfully  used  on  other  types  of  programs,  including  the  C-17 
Globemaster  III  airlifter,  and  the  T-6A  Texan  primary  trainer  programs. 
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these  were  the  only  elements  of  the  overall  programs  that  had  been 
fleshed  out  in  sufficient  technical  detail  to  permit  credible  cost  esti¬ 
mating  efforts.  In  most  of  the  programs  examined,  the  cost  estimates 
for  follow-on  increments  or  spirals  merely  reflected  the  funding  allo¬ 
cations  planned  in  the  Future  Years  Defense  Plan  (FYDP).  As  one 
program  director  noted,  detailed  cost  estimating  on  any  of  these  pro¬ 
grams  beyond  the  first  increment  is  “pure  speculation.” 

This,  of  course,  also  represents  the  potential  political  shortcom¬ 
ing  of  evolutionary  costing.  Such  an  approach  leaves  programs  open 
to  concerns  in  Congress  and  elsewhere  over  contractor  or  service  buy 
in,  especially  in  the  wake  of  such  troubled  programs  as  SBIRS-FIigh. 
In  the  view  of  senior  acquisition  managers  at  SBR,  such  concerns 
drove  cost  analysts  away  from  using  an  evolutionary  costing  ap¬ 
proach,  and  eventually  led  to  an  unfair  penalization  of  the  program 
that  was  rigorously  attempting  to  implement  truth  in  costing. 
Whether  or  not  this  generalization  is  valid  specifically  for  SBR  is  ir¬ 
relevant  here.  The  point  made  by  many  program  managers  that  evo¬ 
lutionary  costing  raises  the  potential  for  political  criticism  is  probably 
justified. 

Fiaving  raised  some  of  the  more  widely  recognized  lessons,  is¬ 
sues,  and  concerns  regarding  EA  that  emerged  from  specific  case 
studies,  the  monograph  now  turns  to  a  brief  summary  review  of  the 
overall  findings  of  the  study. 


CHAPTER  THREE 


Summary  Overview  and  Concluding 
Observations 


Introduction 

Many  of  the  observations  and  policy  recommendations  mentioned  in 
this  chapter  are  applicable  to  all  acquisition  programs,  whether  using 
the  EA  process  or  not.  However,  it  is  the  authors’  view  based  on  both 
case  studies  and  other  independent  assessments  that  EA  tends  to 
promote  certain  conditions  that  raise  issues  for  acquisition  managers 
and  cost  analysts  that,  while  not  unique  to  EA  programs,  tend  to  be 
more  prominent  in  such  programs.  More  is  said  on  this  at  the  end  of 
this  chapter.  This  brief  chapter  summarizes  and  reviews  the  authors’ 
principle  findings  on  evolutionary  acquisition  based  on  a  reading  of 
the  relevant  literature,  examination  of  five  major  space  acquisition 
case  studies,  and  interviews  with  numerous  DoD,  product  center, 
program  office,  industry,  and  other  relevant  acquisition  managers  and 
cost  analysts.  This  chapter  is  divided  into  two  sections.  The  first  pre¬ 
sents  the  authors’  findings  and  insights  that  are  relevant  primarily  to 
acquisition  managers.  The  second  focuses  on  topics  of  special  interest 
to  cost  analysts. 


Summary  of  Acquisition  Management  Findings 

The  new  DoD  guidance  on  EA  permits  greater  flexibility,  but 
does  not  eliminate  conceptual  and  definitional  ambiguity.  A  wide 
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consensus  emerged  among  officials  interviewed  that  the  new  DoD 
5000.1/2  and  NSSAP  03-01  guidance  documents  on  structuring  ac¬ 
quisition  programs  in  general  and  evolutionary  acquisition  programs 
in  particular  permit  far  more  flexibility  in  program  approach  and  im¬ 
plementation  than  was  typical  in  the  past.  Most  program  managers 
considered  this  a  highly  welcome  development,  permitting  them  to 
tailor  program  structures  and  approaches  more  proactively  to  meet 
the  unique  circumstances  of  specific  programs  more  effectively. 

One  of  the  main  reasons  senior  OSD  leadership  issued  the  new 
DoD  5000  acquisition  policy  guidance  in  May  2003  was  the  con¬ 
tinuing  confusion  and  ambiguity  regarding  EA  definitions  and  termi¬ 
nology  that  resulted  from  the  initial,  formal  promulgation  of  EA  as 
the  preferred  acquisition  approach  in  October  2000.  However,  the 
authors’  review  of  the  relevant  published  literature,  five  case  studies, 
and  extensive  interviews  with  acquisition  officials,  suggest  that  there 
are  still  considerable  variations  and  inconsistencies  among  programs, 
even  at  the  same  product  center,  in  the  use  of  EA-related  terminolo¬ 
gies  and  the  application  of  EA  concepts.  For  example,  some  programs 
label  their  EA  segments  “increments,”  and  some  label  them  “spirals,” 
often  apparently  with  little  regard  for  the  significant  distinctions  laid 
out  in  new  DoD  5000  guidance  between  the  spiral  development 
process  and  the  incremental  development  process.  Other  factors,  such 
as  the  use  and  importance  of  feedback  loops,  which  are  of  great  im¬ 
portance  theoretically  in  the  implementation  of  EA,  vary  considerably 
from  program  to  program.  As  discussed  later,  this  lack  of  consistency 
may  be  more  a  function  of  some  major  structural  and  institutional 
challenges  confronting  implementation  of  EA  in  certain  types  of  pro¬ 
grams,  rather  than  confusion  or  ambiguity  regarding  the  definition  of 
the  terms. 

Appropriate  structuring  of  EA  phases  with  operationally  use¬ 
ful  threshold  requirements  and  mapping  the  path  to  overall  ob¬ 
jective  capability  are  major  challenges.  Developing  the  structure, 
phasing,  and  content  of  the  specific  increments  for  EA  programs  is 
proving  to  be  a  major  challenge.  Nearly  all  the  programs  examined 
focused  overwhelmingly  on  the  initial  program  spiral  or  increment. 
Almost  all  the  programs  were  struggling  with  defining  acceptable 
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threshold  requirements  and  capabilities  for  operationally  useful  sys¬ 
tems  for  the  initial  EA  phase,  as  well  as  for  follow-on  phases.  Objec¬ 
tive  end  requirements,  and  sometimes  even  objective  end  capabilities 
for  the  overall  program,  often  remained  vague  and  highly  speculative. 

Often  much  uncertainty  existed  even  for  the  initial  Phase  1  in¬ 
crement  or  spiral.  The  first  or  initial  phases  for  virtually  all  the  pro¬ 
grams  examined  appear  to  be  evolving  toward  de  facto  stand-alone 
programs.  In  many  of  the  programs,  acquisition  managers  experi¬ 
enced  strong  pressure  from  user  communities  to  maximize  capability 
targets  for  the  initial  segment,  thus  pushing  the  first  phase  increas¬ 
ingly  toward  something  approaching  a  traditional  single-step-to- 
capability  program  with  internal  block  upgrades. 

The  use  of  the  spiral  development  process  on  major  hard¬ 
ware  acquisition  programs  raises  serious  challenges  for  program 
managers  in  the  current  acquisition  environment.  The  very  uncer¬ 
tainties  that  provide  acquisition  managers  with  valuable  flexibility 
necessary  to  gain  the  expected  benefits  from  EA  through  spiral  devel¬ 
opment  also  raise  considerable  challenges  for  managers  in  the  existing 
acquisition  environment.  This  is  particularly  true,  as  discussed  in  the 
previous  chapter,  in  large,  high-profile  programs  such  as  SBR,  the 
original  Big  RAIDRS,  and,  to  a  lesser  extent,  KEI.  It  is  not  always 
politically  feasible  to  expect  Congress  to  fully  support  potentially  very 
costly  programs  based  on  new  and  possibly  immature  technologies 
and  concepts — programs  that  may  not  be  as  highly  structured  as  tra¬ 
ditional  programs,  and  that  do  not  have  precise  and  detailed  concepts 
of  the  first  phase  threshold  system  requirements  and  capabilities, 
much  less  the  final  end-phase  objective  system.  This  is  particularly 
true  of  large,  transformational  space  programs,  given  the  recent  un¬ 
happy  experience  with  SBIRS  High  and  other  earlier  efforts. 

Acquisition  managers  of  such  programs  report  that  they  are 
subjected  to  very  strong  pressure  from  the  requirements  and  cost 
analysis  community  to  provide  far  more  detail  about  the  end  stages  of 
the  program  than  they  believe  is  feasible  under  the  spiral  approach. 
The  whole  spiral  concept  assumes  that  the  major  components  of  the 
program  emerge  and  are  defined  through  experimentation  as  the  pro¬ 
gram  progresses.  Such  a  concept  may  be  too  open-ended  to  be  politi- 
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cally  practical  in  the  overheated  environment  of  defense  budget  poli¬ 
tics.  Useful  flexibility  and  prudent  incrementalism  to  one  observer 
may  appear  to  another  observer  to  be  a  devious  attempt  to  buy-in. 

The  authors  found  that  many  of  the  large,  higher-profile  pro¬ 
grams  examined,  such  as  SBR,  GPS  III,  and  Big  RAIDRS,  were 
evolving  away  from  the  use  of  the  spiral  development  process.  These 
programs  were  increasingly  focused  on  clearly  structuring  the  initial 
increment  in  considerable  detail,  and  in  nailing  down  and  finalizing 
both  threshold  and  objective  requirements  for  the  first  increment.  In 
the  process  of  doing  spiral  development  (SD),  some,  such  as  SBR, 
came  under  great  budgetary  pressure  from  Congress,  and  were  re¬ 
structured  or  evolved  toward  something  approaching  technology 
demonstration  programs  or  more  traditional  single-step-to-capability 
programs  with  stepped  block  upgrades. 

It  is  no  wonder,  then,  that  in  part  due  to  the  experiences  of  SBR 
and  other  programs,  senior  Air  Force  officials  and  acquisition  manag¬ 
ers  appear  to  have  steadily  retreated  from  openly  advocating  the  spiral 
development  process  and  moved  increasingly  toward  incremental  de¬ 
velopment.  Yet  the  risk  in  moving  toward  incremental  development 
is  that  programs  using  such  an  approach  may  easily  transform  into  de 
facto  traditional  single-step-to-capability  programs,  perhaps  with  in¬ 
ternal  block  upgrades,  that  are  indistinguishable  from  the  old  acquisi¬ 
tion  approaches. 

Based  on  these  case  studies,  it  appears  that  a  spiral  development 
approach  is  more  politically  practical  on  smaller,  more  modest,  lower- 
profile  programs  such  as  the  current  RAIDRS  and,  to  a  lesser  extent, 
SBSS.  With  a  smaller  likelihood  of  being  subjected  to  severe  public 
scrutiny  and  criticism  over  the  lack  of  precise  objective  program  end 
requirements,  these  programs  have  greater  freedom  to  pursue  a  more 
flexible  approach  to  evolving  requirements  over  time.  Yet  even  in 
these  programs,  the  tendency  has  clearly  been  to  move  toward  more 
precise  and  early  definition  of  final  requirements,  in  part  due  to  pres¬ 
sures  from  the  cost  estimating  community  and  the  need  to  meet  a 
variety  of  regulatory  oversight  and  reporting  requirements. 

Therefore,  the  authors  believe  that  evolutionary  acquisition  us¬ 
ing  the  preferred  approach  of  spiral  development,  as  laid  out  in  the 
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most  recent  DoD  5000  and  NSSAP  03-01  guidance,  may  be  difficult 
to  implement  in  the  current  political  and  acquisition  environment  on 
major  DoD  space  programs,  and  possibly  on  other  large-scale  DoD 
hardware  acquisition  programs.1  At  best,  EA  using  spiral  develop¬ 
ment  may  be  one  useful  tool  that  can  be  used  in  some  limited  cir¬ 
cumstances  on  software  programs,  on  smaller-scale  hardware  pro¬ 
grams,  or  perhaps  on  programs  such  as  KEI  that  operate  outside  the 
traditional  acquisition  framework. 


Summary  of  Cost  Management  Findings 

EA  poses  few  insurmountable  challenges  to  cost  analysts.  Com¬ 
pared  to  other  traditional  program  approaches,  EA  requires  more 
extensive  up-front  and  continuing  participation  of  the  cost  analy¬ 
sis  community  as  the  program  evolves.  Based  on  the  authors’  exten¬ 
sive  interviews  with  senior  cost  analysts  at  the  Air  Force  Cost  Analysis 
Agency  (AFCAA),  the  CAIG,  and  SMC,  the  authors  have  concluded 
that  while  EA  substantially  increases  the  up-front  and  ongoing 
workload  for  cost  analysts,  it  in  and  of  itself  raises  few  insurmount¬ 
able  problems  or  obstacles  for  cost  analysts.  The  most  challenging 
issue  is  having  access  to  realistic  and  independent  technical  assess¬ 
ment  of  the  program  baseline  assumptions.  NSSAP  03-01  in  par¬ 
ticular  places  a  very  heavy  emphasis  on  early,  high-quality,  and  mul¬ 
tiple  independent  cost  analyses  and  other  forms  of  program  review  for 
all  major  space  programs.  This  results  in  a  considerable  increase  in 
workload  for  cost  analysts. 

There  was  a  consensus  among  the  program  managers  and  cost 
analysts  interviewed  that  the  only  sensible  approach  to  dealing  with 
EA  programs  was  to  adopt  what  amounted  to  an  evolutionary  costing 
strategy  to  mimic  the  evolutionary  nature  of  the  development  of  the 
system.  As  the  system  design  and  technology  evolve  and  mature 


1  A  comprehensive  analysis  of  the  effects  of  EA  on  nonspace  MDAPs  is  necessary  to  make  a 
more  definitive  statement. 
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through  various  spirals  or  increments,  so  must  the  cost  estimates  go 
through  cycles  of  increasing  refinement  and  fidelity. 

Another  key  strategy  for  coping  with  cost  estimating  on  EA  pro¬ 
grams  is  to  focus  heavily  on  the  initial  spiral  or  increment,  whose  spe¬ 
cific  content  is  generally  far  better  defined  than  later  spirals  or  incre¬ 
ments.  Typically,  follow-on  increments  are  nominally  costed  to  the 
Future  Years  Defense  Plan,  and  estimates  are  revised  as  requirements 
are  refined.  As  one  program  manager  noted,  costing  future  incre¬ 
ments  or  spirals  on  EA  programs  is  often  an  exercise  in  “pure  specula¬ 
tion.”  While  cost  analysts  appear  to  be  having  no  problem  adjusting 
to  this  approach,  evolutionary  costing  can  increase  program  exposure 
to  political  allegations  of  buy-in  or  lack  of  full  disclosure. 

Concerns  linger  about  the  ability  of  EA  programs  to  generate 
credible  early  program  LCC  estimates  when  future  requirements 
and  technologies  remain  so  uncertain.  Overall,  most  cost  analysts 
interviewed  expressed  generally  positive  views  about  EA.  Nonetheless, 
lingering  concerns  did  surface  during  interviews  regarding  a  variety  of 
cost  issues  associated  with  EA:2 

1.  Committing  the  USAF  to  large,  costly  programs  before  the  full 
cost  implications  of  the  program  are  well  understood 

2.  Accurately  assessing  total  program  LCC,  support  costs,  and  retro¬ 
fit  costs  based  on  sound  and  independent  technical  assessment  of 
the  program  baseline 

3.  Adequately  budgeting  for  the  potentially  high  variability  of  cost 
outcomes  arising  from  the  high  degree  of  uncertainty  surrounding 
inputs  to  cost  models. 

4.  Accurately  accounting  for  the  potential  cost  implications  of  re¬ 
quirements  creep  arising  from  multiple  users  and  planned  inser¬ 
tion  of  technologies  of  uncertain  future  maturity. 

The  cost  analysts  interviewed  did  not  view  any  of  these  potential 
challenges  as  show  stoppers.  Nonetheless,  there  was  recognition — at 


2  Concerns  similar  to  those  listed  here  have  been  expressed  in  many  quarters.  For  example, 
see  Levin  (2003). 
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least  among  some  of  those  interviewed — that  EA  would  increase  the 
cost  analysts’  initial  workload  and  require  substantial  interface  with 
the  engineers  and  the  contractor.  In  addition,  some  of  these  issues 
could  pose  serious  political  problems  for  program  managers.  In  addi¬ 
tion,  all  the  programs  surveyed  were  in  such  an  early  stage  that  several 
of  these  challenges  were  only  latent. 

For  example,  the  challenge  of  estimating  the  program  cost  im¬ 
plications  for  logistics  support  of  multiple  different  versions  of  the 
same  system  arising  from  different  increments  or  spirals  (none  of 
which  can  be  predicted  accurately  at  program  inception),  and  the 
probable  need  to  retrofit  or  upgrade  earlier  versions  to  bring  them  up 
to  the  standard  of  the  more  recent  variants  later  in  the  program,  were 
recognized  as  potentially  significant  cost  estimating  issues. 

Clearly  the  experience  of  the  SBR  and  KEI  programs  demon¬ 
strate  that  the  use  of  EA,  especially  using  the  spiral  development 
process,  greatly  increases  exposure  to  the  possibility  of  criticism  from 
Congress  and  other  outside  observers,  at  least  in  situations  where 
technologies  may  be  new  and  immature,  or  where  disagreement  exists 
over  CONOPS  and  mission  effectiveness. 

Nonetheless,  a  strong  consensus  emerged  from  these  interviews 
with  cost  analysts  that  EA  is  an  important  and  useful  tool  that  pro¬ 
vides  program  managers  with  increased  flexibility,  and  that  can  be 
accommodated  adequately  by  the  cost  community  through  the  use  of 
evolutionary  costing. 


Applicability  of  EA  Findings  from  Space  Programs  to 
Other  DoD  Acquisition  Programs 

As  discussed  in  Chapter  One,  space  programs  differ  from  other  types 
of  DoD  defense  acquisition  programs.  This  raises  the  legitimate  ques¬ 
tion  of  how  relevant  EA  findings  drawn  from  space  program  case 
studies  are  to  other  types  of  defense  acquisition  programs.  In  Chapter 
One  the  authors  examined  the  major  differences  between  DoD  space 
and  other  types  of  programs  and  made  some  comparisons  between 
the  formal  acquisition  policy  documents  governing  acquisition  of  the 
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two  categories  of  programs.  In  it,  the  authors  determined  that  this 
information  was  inconclusive  regarding  the  applicability  of  EA  find¬ 
ings  based  on  space  programs  to  other  types  of  programs. 

In  this  subsection,  the  authors  examine  information  that  appears 
to  shed  more  light  on  the  issue:  other  systematic  studies  of  EA  im¬ 
plementation  on  different  types  of  programs,  and  past  and  ongoing 
RAND  research  on  several  UAV  programs  that  many  in  DoD  have 
long  considered  key  test  cases  for  the  EA  strategy. 

Other  Research  Findings 

The  most  extensive  published  study  of  which  the  authors  are  aware 
that  examines  actual  EA  implementation  issues  based  on  real-world 
experience  with  nonspace  programs  is  a  master’s  thesis  written  by  an 
Air  Force  officer,  Gary  L.  Wellman,  in  the  Acquisition  Management 
Program  in  the  Department  of  Systems  and  Engineering  Manage¬ 
ment  at  the  Air  Force  Institute  of  Technology,  Air  University,  Day- 
ton,  Ohio  (Wellman,  2003).  Wellman  conducted  structured  group 
discussions  with  civilian  and  military  contracting  officers  located  at 
three  different  Air  Force  product  centers:  Air  Armament  Center 
(AAC),  Eglin  Air  Force  Base,  Fla.;  Aeronautical  Systems  Center 
(ASC),  Wright-Patterson  Air  Force  Base,  Ohio;  and  Electronic  Sys¬ 
tems  Center  (ESC),  Hanscom  Air  Force  Base,  Mass.  While  this  study 
focused  exclusively  on  SMC  plus  MDA,  Wellman’s  study  examined 
EA  issues  at  the  other  three  principal  Air  Force  product  centers. 
Wellman  focused  primarily  on  contracting  issues,  and  interviewed 
only  contracting  professionals  who  had  experience  with  EA  programs. 
However,  his  larger  goal  was  “to  identify  potential  roadblocks  to  im¬ 
plementation  of  Evolutionary  Acquisition  strategies,”  and  identifying 
approaches  that  “may  counter  the  identified  challenges”  (Wellman, 
2003,  p.  viii). 

Wellman  undertook  his  study  because,  while  “EA  has  been 
mandated  as  the  acquisition  strategy  for  all  ACAT  I  programs,  .  .  . 
[n]o  firm  contracting  guidance  has  been  provided  on  how  to  accom¬ 
plish  this  task  either  at  the  DoD  or  the  individual  service  level” 
(Wellman,  2003,  p.  6).  The  authors  found  that  this  situation  existed 
across  the  board  on  most  EA  implementation  issues,  and  had  con- 
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tributed  to  the  lack  of  consistency  and  confusion  regarding  the  im¬ 
plementation  of  the  policy. 

Wellman’s  findings  from  the  three  product  centers  can  be  sum¬ 
marized  as  follows: 

1.  The  “increment  definition  process”  is  identified  as  a  serious  area 
of  “concern,”  while  “unknown  capabilities”  in  future  increments 
“pose  their  own  set  of  unique  challenges  in  the  refinement  of  fu¬ 
ture  increments”  (Wellman,  2003,  p.  69).  More  specifically,  in¬ 
terviewees  identified  a  variety  of  challenges  in  this  area:  establish¬ 
ing  a  process  and  an  approach  to  defining  each  increment; 
defining  future  increments  when  capabilities  are  unknown;  de¬ 
termining  who  should  be  in  charge  of  the  process;  and  other  is¬ 
sues.  Interviewees  stressed  that  increments  and  the  end  capabilities 
and  systems  should  be  well  defined  at  the  very  beginning  of  an  in¬ 
crement,  but  often  are  not  (Wellman,  2003,  pp.  68-69). 

2.  Serious  constraints  are  identified  that  inhibit  the  successful  im¬ 
plementation  of  EA  strategies.  These  include  “mindset  con¬ 
straints”  (under  which  the  author  includes  “lack  of  specific  guid¬ 
ance”  from  management);  “partition  constraints”  (in  which  the 
author  apparently  includes  regulatory  issues);  and  “brick  wall  con¬ 
straints,”  which  are  the  most  serious  political,  statutory,  and 
regulatory  constraints.3  By  far  the  most  important  of  these  is  “the 
government  funding  process.”  Here  the  most  significant  con¬ 
straints  were  “the  affordability  of  the  increments,”  the  “availability 
and  timing  of  funds,”  and  the  “color  of  money”  (Wellman,  2003, 
p.  45).  Other  difficult  areas  included  a  range  of  statutory  and 
regulatory  roadblocks,  such  as  the  Competition  in  Contracting 
Act  and  the  Small  Business  Act.  Wellman  concludes  that  “in  order 
to  effectively  implement  EA,  the  Government  must  ensure  that  its 
practices  and  processes,  to  include  regulatory  guidance,  reflect  the 
intent  of  the  preference  for  EA  strategies”  (Wellman,  2003, 
p.  74). 


3  “Brick  wall”  constraints  are  ominously  defined  as  “those  that  cannot  be  overcome  and  are 
considered  unalterable”  (Wellman,  2003,  p.  43). 
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While  Wellman  is  primarily  concerned  with  the  minutiae  of 
contracting,  and  spends  most  of  his  efforts  at  examining  detailed  con¬ 
tracting  workarounds  for  specific  practical  contracting  issues,  his 
broader  conclusions  seem  to  be  consistent  with  the  authors’  findings. 
Most  importantly,  his  respondents  identified  the  key  challenge  of  the 
“increment  identification  process,”  and  the  problem  of  defining  fu¬ 
ture  increments  with  unknown  capabilities.  This  was  perhaps  the  cen¬ 
tral  problem  area  for  most  of  the  space  programs  examined  here,  par¬ 
ticularly  SBR,  RAIDRS,  and  GPS  III. 

The  second  publicly  available  study  is  perhaps  even  more  rele¬ 
vant  to  this  research,  and  strongly  confirms  many  of  the  authors’ 
findings  from  space  programs.  This  is  a  case  study  conducted  by  Walt 
Pingel,  a  cost  analyst  at  the  Global  Hawk  UAV  Program  Office  at 
ASC  who  has  worked  on  this  program  from  its  earliest  phases,  as  well 
as  other  UAV  programs  such  as  DarkStar.  The  case  study  is  specifi¬ 
cally  aimed  at  evaluating  the  implications  of  EA  for  cost  analysts.  The 
case  study  findings  were  presented  to  the  DoD  Cost  Analysis  Sympo¬ 
sium  in  January  2003  (Pingel,  2003). 

Global  Hawk  is  a  complex  program  with  many  unique  and  un¬ 
usual  program  attributes.  This  is  not  the  place  for  a  detailed  review  of 
this  program.  Nonetheless,  as  noted  in  Chapter  One,  Pingel  correctly 
argues  that  Global  Hawk  has  historically  been  viewed  as  a  principal 
test  case  for  the  efficacy  of  EA.4  Therefore,  it  seems  appropriate  to 
summarize  his  key  findings  for  the  purpose  of  demonstrating  their 
similarity  to  the  authors’  findings  on  space  programs,  rather  than  to 
make  a  judgment  of  overall  performance  of  the  Global  Hawk  pro¬ 
gram.  Pingel’s  findings  can  be  summarized  as  follows: 

1 .  “  [The]  spiral  development  process  engenders  a  very  high  amount 
of  concurrency  between  program  phases.  ...  A  high  degree  of  co- 


4  For  another  example  of  the  use  of  Global  Hawk  as  a  key  DoD  case  study,  see  Johnson  and 
Johnson  (2002).  A  current  research  project  undertaken  by  the  National  Academies  titled 
“Test  Strategies  for  Evolutionary  Defense  Acquisition”  has  looked  at  Global  Hawk  as  a  key 
case  study,  along  with  JASSM,  JSOW,  Small  Diameter  Bomb,  and  FCS.  Of  these  cases,  only 
Global  Hawk  is  a  major  system  platform  program  that  is  well  advanced  in  the  procurement 
cycle,  and  is  already  in  production. 
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ordination  is  required  at  all  times  to  ensure  the  program  strategy 
does  not  become  disconnected  between  acquisition  phases.” 
Pingel  (2003,  p.  12)  then  notes,  “the  program  estimator  is  one  of 
the  people  who  pays  a  high  price  for  disconnected,  obtuse,  or  con¬ 
flicting  program  strategy  and  assumptions.” 

2.  The  Global  Hawk  program  using  EA  has  been  characterized  by 
“severe  swings  in  force  structure,  acquisition  strategy,  schedule, 
and  operational  concept.”  This  situation  has  “meant  seismic  shifts 
in  the  nature  and  composition  of  the  program  estimate.”5 

3.  “The  concept  of  spiral  development  is  one  of  many  increments,  all 
being  independently  managed.  In  reality  however,  there  are  cer¬ 
tain  activities  that  are  Level  of  Effort  in  nature,  that.  .  .  continue 
across  multiple  spirals  and  service  a  host  of  spiral  increments.  So 
the  estimator  faces  the  challenge  of  not  only  estimating  each  inde¬ 
pendent  development  activity,  but  the  ‘overlay  structure’  sur¬ 
rounding  those  activities”  (Pingel,  2003,  p.  14). 

4.  A  key  “problem  is  that  many  of  the  [spiral  increments]  are  poorly 
defined.”  While  this  problem  may  also  arise  on  traditional  pro¬ 
grams,  spiral  development  adds  “another  wrinkle.  .  .  .  That  wrin¬ 
kle  is  the  fact  that  the  user,  through  continuous  feedback,  con¬ 
stantly  reprioritizes  the  scope  and  content  of  the  spirals,  usually 
future  spirals,  but  sometimes  even  for  increments  on  contract.”  As 
a  result,  “the  estimator  constantly  has  to  adjust  the  increment  es¬ 
timates  that  are  impacted  by  the  change,  and  must  understand  the 
synergies  between  cost  elements  so  that  overlay  activities.  .  .  can  be 
adjusted  accordingly”  (Pingel,  2003,  p.  14). 

5.  “Because  of  the  rate  of  change  in  a  spiral  program,  the  regulatory 
review  system  struggles  to  keep  up  with  changes  in  both  program 


5  Pingel  (2003,  p.  13).  At  least  some  of  this  is  due  to  the  urgent  post-9/ 1 1  requirements  to 
field  UAV  capabilities  rapidly  in  Afghanistan  and  Iraq.  DoD  deployed  prototype  versions  of 
both  Global  Hawk  and  Predator  that  had  not  been  fully  tested  to  active  theaters  of  opera¬ 
tions.  These  actions  caused  major  acquisition  program  disruptions,  and  significantly  in¬ 
creased  the  flow-back  of  requirements  and  technical  changes  based  on  field  experience. 
Nonetheless,  both  contractor  and  government  officials  strongly  believe  that  EA  and  spiral 
development  approaches  on  their  own  inherently  contribute  to  program  flux  and  instabili¬ 
ties,  particularly  in  the  area  of  requirements  change  and  growth. 
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content  and  direction.  Our  current  regulatory  system  is  simply 
not  designed  to  deal  with  a  program  that  changes  constantly  and 
swiftly.  .  .  .  Valuable,  limited  resources  and  time  are  spent  re¬ 
viewing  and  re-reviewing  previous  analysis,  re-educating  manage¬ 
ment,  and  re-justifying  program  direction  instead  of  performing 
current  data  collection,  analysis,  as  well  as  the  general  business  of 
running  a  program.”  While  Pingel  notes  that  “many  programs 
have  these  problems,”  he  insists  that  “the  conflict  between  the  na¬ 
ture  and  speed  of  a  spiral  development  versus  our  current  regula¬ 
tory  environment”  makes  the  problem  far  worse  in  an  EA  pro¬ 
gram  (Pingel,  2003,  p.  11). 

6.  And  finally,  Pingel  concludes  that  “if  there  is  anything  you  should 
take  away  as  a  lesson  learned  it’s  that  logistics,  retrofit,  and  inte¬ 
gration  activities’  costs  are  very  difficult  to  ascertain,  and  have  the 
potential  of  becoming  much  more  expensive  than  historical  data 
would  indicate”  (Pingel,  2003,  p.  17). 

Ongoing  RAND  Research  on  UAVs 

As  mentioned  in  Chapter  One,  RAND  has  published  several  studies 
in  the  past  on  the  Global  Hawk  and  Predator  acquisition  programs, 
and  is  currently  conducting  ongoing  research  on  these  and  other 
UAV  programs.  While  that  work  is  still  underway,  is  focusing  on 
other  issues,  and  has  not  yet  generated  final  research  results,  the  in¬ 
formation  RAND  has  collected  through  the  summer  of  2005  on  the 
EA  strategies  used  on  Global  Hawk  and  other  UAV  programs  is  con¬ 
sistent  with  the  findings  of  the  Pingel  study. 

For  the  Global  Hawk  program,  RAND  has  conducted  extensive 
interviews  with  program  management  and  cost  professionals  repre¬ 
senting  both  the  Air  Force  Program  Office  and  the  prime  contractor, 
as  well  as  other  entities.  Many  of  the  RAND  interim  findings  seem  to 
confirm  Pingel’s  observations.  Some  of  the  key  points  that  have 
emerged  to  date  relating  to  the  implementation  of  the  EA  strategy 
include  the  following: 

1.  Program  managers  and  cost  analysts  have  faced  major  challenges 
caused  by  the  constant  changes  in  requirements,  operational  con- 


Summary  Overview  and  Concluding  Observations  101 


cepts,  and  technical  specifications.  It  is  the  consensus  view  among 
both  prime  contractor  and  government  personnel  that  these  chal¬ 
lenges  are  made  much  more  daunting  by  the  early  fielding  and 
constant  requirements  feedback  process  promoted  by  EA.  To 
paraphrase  one  contractor  official,  the  EA  process  provides  an  irre¬ 
sistible  opportunity  and  temptation  for  the  user  to  constantly 
change,  add,  and  fine-tune  requirements  and  operational  con¬ 
cepts.  While  this  is  not  necessarily  a  bad  thing,  especially  from  the 
user’s  perspective,  it  has  created  continuous  change  and  uncer¬ 
tainty  in  all  aspects  of  program  management  and  cost  analysis.  Ac¬ 
cording  to  the  Global  Elawk  prime  contractor,  the  program  has 
experienced  unprecedented  levels  of  “requirements  churn.”  In  cal¬ 
endar  year  2004  alone,  the  contractor  experienced  on  the  order  of 
200  contract  actions,  contributing  significantly  to  program  cost 
growth.  In  addition,  the  large  numbers  of  contract  actions  placed 
an  unusually  high  workload  burden  on  both  the  contractor’s  en¬ 
gineering  staff  and  the  cost  analysts,  and  diverted  them  from  fo¬ 
cusing  on  core  aspects  of  the  program.  The  whole  requirements 
and  technical  change  process  has  proven  extremely  difficult  to  dis¬ 
cipline  and  control. 

2.  Defining  the  precise  technical  content  of  specific  spirals  or  incre¬ 
ments  and  costing  them  appropriately  has  posed  challenges.  The 
ad  hoc  transfer  of  specific  subelements  and  tasks  from  one  spiral 
to  another  has  complicated  this,  as  program  goals,  funding,  and 
technology  maturity  assessments  constantly  change.  Another 
problem  has  been  allocating  overhead  and  other  aspects  of  the 
“overlay  structure”  among  different  spirals.  To  paraphrase  one 
contracting  official,  EA,  with  its  multiple,  overlapping  increments, 
opens  the  door  to  much  more  complexity  and  change  than  is 
common  in  typical  traditional  programs. 

3.  A  consensus  view  appears  to  exist  among  Global  Hawk  program 
office  acquisition  managers  and  cost  analysts  that  EA  and  spiral 
development  do  result  in  fielding  operationally  useful  systems 
much  sooner  than  the  traditional  approach,  but  at  a  much  higher 
cost.  These  officials  argue  that  a  cost-effective  program  requires 
stable  requirements,  system  configuration,  and  quantities,  and 
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adequate  funding.  In  their  view,  EA  and  spiral  development  ap¬ 
proaches  promote  constant  flux  in  all  these  program  attributes, 
leading  inevitably  to  cost  estimating  difficulties  and  cost  growth. 
The  key  lesson  learned  from  Global  Hawk,  according  to  one  offi¬ 
cial,  is  that  the  only  way  to  implement  spiral  development  effec¬ 
tively  was  to  provide  unlimited  funding  to  cover  the  unending 
changes.  Otherwise,  in  the  colorful  words  of  another  official,  spi¬ 
ral  development  tends  to  end  up  becoming  “spiral  death.” 

Based  on  this  quick  review  of  two  other  independent  studies  of 
the  implementation  of  EA  strategies  on  nonspace  programs,  as  well  as 
ongoing  RAND  research  on  Global  Hawk  and  other  UAVs,  the 
authors  conclude  that  the  findings  reported  in  this  monograph  based 
on  space  program  case  studies  are  reasonably  robust,  and  are  largely 
applicable  to  other  major  DoD  defense  acquisition  programs.  The 
authors  find  that  all  the  cases  reviewed,  including  those  in  the  Well¬ 
man  study,  which  covered  a  wide  range  of  programs  at  ASC,  AAC, 
and  ESC;  the  Pingel  case  study  of  Global  Hawk;  and  RAND’s  own 
past  and  ongoing  research  on  Global  Hawk  and  other  UAVs,  have 
found  with  remarkable  consistency  that  the  real-world  application  of 
EA  approaches  raises  serious  challenges  for  program  managers  and 
cost  analysts  in  the  same  set  of  areas.  While  these  challenges  are  not 
unique  to  programs  using  an  EA  approach,  they  appear  to  be  more 
formidable  in  EA  programs.  These  areas  include  the  following: 

•  Requirements  and  technology  churn 

•  Increment  or  spiral  definition  and  content 

•  Program  complexity  and  concurrency 

•  Logistics  planning  and  complexity 

•  Funding  coordination  for  increments 

•  Regulatory  environment  and  oversight  requirements. 

To  be  fully  confident  in  the  robustness  of  these  findings,  how¬ 
ever,  the  authors  believe  that  in  the  future,  extensive  additional  case 
study  analysis  of  nonspace  EA  programs  is  fully  warranted. 
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EA  Challenges:  A  Flawed  Approach,  or  Flawed 
Implementation? 

Based  on  the  case  studies  and  on  other  published  assessments,  there 
appears  to  be  little  doubt  that  EA  strategies,  whatever  benefits  they 
may  produce,  have  resulted  in  significant  challenges  for  acquisition 
managers  and  cost  analysts.  It  is  legitimate  to  question,  however, 
whether  these  challenges  arise  inherently  from  the  strategy  itself,  or 
rather  are  more  a  function  of  incomplete  or  improper  implementa¬ 
tion  of  the  strategy. 

While  a  precise,  quantitative  answer  to  this  question  is  not  pos¬ 
sible,  it  is  the  authors’  sense  that  the  challenges  posed  by  EA  strategies 
arise  from  a  combination  of  both  causes. 

Probably  the  single  most  daunting  challenges  posed  by  EA  are 
controlling  requirements  and  technology  churn,  and  stabilization  of 
the  definition  and  content  of  increments.  Based  on  the  review  in 
Chapter  One  of  the  new  DoDI  5000.2  documents  and  the  new 
NSSAP  03-01,  it  is  clear  that  the  authors  of  these  documents  were 
extremely  sensitive  to  these  challenges  and  explicitly  restructured  the 
proscribed  acquisition  process  in  order  to  try  to  ensure  that  require¬ 
ments  were  well  defined,  selected  technologies  were  mature,  and  the 
structure  of  a  given  increment  was  well  thought  out  prior  to  formal 
program  initiation  and  passage  through  Milestone  or  KDP  B. 

In  the  real  world,  however,  it  has  proven  difficult  to  implement 
this  carefully  planned  structure.  The  reason  in  part  seems  to  be  that 
the  EA  approach,  with  its  strong  emphasis  on  user  feedback,  and  its 
flexible  approach  to  final  requirements,  inherently  encourages  strong 
pressures  from  user  communities  to  continue  changing  requirements 
and  redefining  increments  throughout  the  acquisition  process.  This  is 
true  in  the  early  phases  of  space  programs  examined  such  as  SBR, 
where  acquisition  managers  were  bedeviled  from  the  start,  and  with¬ 
out  let-up,  by  constantly  shifting  requirements  and  capabilities  de¬ 
mands  from  user  communities.  Global  Hawk,  and,  to  a  lesser  extent 
Predator,  which  are  totally  different  kinds  of  programs,  and  which  are 
in  a  far  more  advanced  stage  of  the  acquisition  process,  have  both  ex¬ 
perienced  the  same  kind  of  unusually  high  degrees  of  requirements 
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and  technology  churn.  Thus,  although  safeguards  against  these  prob¬ 
lems  were  carefully  written  into  the  new  acquisition  regulations,  the 
EA  concept  itself  seems  to  promote  the  uncertainties  and  churn  that 
cause  the  challenges  for  acquisition  managers  and  cost  analysts. 

Increased  program  complexity  was  identified  by  all  interviewees 
in  the  RAND  case  studies,  as  well  in  other  independent  published 
studies,  as  an  important  challenge  posed  by  EA  strategies.  There  is  no 
question  that  increased  program  complexity  is  an  inherent  attribute 
of  the  EA  approach.  This  is  because  EA  envisions  multiple  incre¬ 
ments,  all  of  which  are  treated  in  a  management  sense  as  quasi¬ 
separate  programs,  with  their  own  milestone  reviews,  oversight 
documentation,  and  so  forth.  This  complexity  is  increased  by  the 
tendency  to  move  content  around  from  one  increment  to  another,  a 
tendency  that  appears  common  on  both  space  programs  in  their  early 
ph  ases  and  other  types  of  programs  such  as  Global  Hawk  that  are  far 
more  advanced  in  the  acquisition  process. 

Given  the  challenges  posed  by  the  difficulty  in  defining  incre¬ 
ments,  the  constant  requirements  feedback  and  churn,  and  the  diffi¬ 
culties  encountered  in  structuring  and  defining  the  content  of  specific 
increments  or  spirals,  it  is  not  surprising  that  planning  funding  for 
increments  has  also  proven  to  be  a  significant  challenge  inherent  to 
the  EA  approach. 

Finally,  there  is  little  doubt  that  the  overall  acquisition  regula¬ 
tory  and  oversight  environment  affects  the  ability  to  implement  EA  in 
the  form  initially  envisioned  by  its  original  advocates.  The  extent  to 
which  the  challenges  that  have  arisen  in  EA  programs  are  caused  by 
the  traditional  acquisition  regulatory  environment  is  unclear.  The 
new  DoDI  5000.2  and  NSSAP  03-01  greatly  increase  the  flexibility 
available  to  program  managers.  On  the  other  hand,  a  significant  body 
of  regulatory  and  statutory  requirements  based  on  more  traditional 
approaches  still  exists.  As  just  one  example,  the  requirement  to  de¬ 
velop  full  and  accurate  LCC  estimates  in  the  early  stages  of  the  acqui¬ 
sition  process  is  very  difficult  to  achieve  in  an  EA  program  using  a 
spiral  approach,  and  indeed  is  not  in  line  with  the  basic  philosophy 
underlying  spiral  development. 
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Finally,  there  is  no  question  but  that  the  EA  strategies  inherently 
complicate  logistics  planning  and  cost  estimating.  By  focusing  on 
space  programs,  this  study  may  have  underemphasized  this  challenge. 
The  indications  from  ongoing  studies  of  Global  Hawk,  Predator,  and 
other  UAVs  using  spiral  approaches  suggest  that  detailed  logistics 
planning  and  accurate  logistics  cost  estimating  are  inherently  compli¬ 
cated  by  the  use  of  EA,  because  this  approach  produces  so  many  vari¬ 
ants  of  the  basic  system. 

Therefore,  for  the  most  part,  it  appears  that  most,  but  not  all,  of 
the  significant  challenges  posed  by  EA  to  the  acquisition  and  cost 
analysis  communities  are  inherent  in  the  process  itself.  It  does  not 
follow  that  therefore  the  process  of  evolutionary  acquisition  is  inher¬ 
ently  flawed  and  should  be  rejected.  Indeed,  it  could  be  argued  that 
from  the  perspective  of  the  warfighter,  EA  represents  a  significant 
improvement  over  past  acquisition  approaches.  And  of  course  the  ac¬ 
quisition  community  exists  only  to  serve  the  warfighter.  Nonetheless, 
program  managers  and  cost  estimators  need  to  anticipate  and  be  pre¬ 
pared  to  cope  with  the  increased  challenges  posed  by  EA  when  it  is 
selected  as  the  core  acquisition  strategy  for  a  major  program.  This  is 
because  the  acquisition  community  is  also  responsible  to  the  taxpayer 
and  must  realistically  adapt  to  the  inevitable  political  realities  within 
which  the  acquisition  process  operates. 
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